Display apparatus for displaying image data received from an image pickup apparatus attached to a moving body specified by specification information

ABSTRACT

The present invention makes it possible to let a user arbitrarily and easily see any other scene than himself or herself in his or her view (the scene seen from any other moving body than him or her). In a display device (an image pickup and display device  1  or a display device  40 ), specification information is generated to specify a specific image pickup device of outer image pickup devices (the image pickup and display device  1  or and image pickup device  30 ), and image data are received from the image pickup device specified by the specification information and are displayed. As an outer image pickup device, for example, and image pickup device wore by some other person, image pickup devices set at a car, an electric train, and the like, or further mage pickup devices set at an animal, a bird, and the like are supposed. Image data picked up by these image pickup devices are transmitted to the display side and the image display is carried out in the display device. Thus, a user of the display device can see images of viewing scenes watched by other people, for example, images of viewing scenes seen from a car, an electric train, and the like, or images of viewing scenes seen from an animal, a bird, and the like.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/212,952, filed Jul. 18, 2016, which is a continuation of U.S.application Ser. No. 12/515,747, filed Aug. 3, 2009, which is now U.S.Pat. No. 9,413,983, and is based upon and claims the benefit of priorityto International Application No. PCT/JP07/071564, filed Nov. 6, 2007,and from the prior Japanese Patent Application No. 2006-315750, filedNov. 22, 2006 and Japanese Patent Application No. 2007-039902, filedFeb. 20, 2007. The entire contents of each of these documents areincorporated herein by reference.

TECHNICAL FIELD

The present invention relates generally to an image display system, adisplay apparatus, and a display method and, more particularly, to atechnology configured to display, on the side of the display apparatus,images taken by an external image pickup apparatus.

BACKGROUND ART

Japanese Patent Laid-open No. 2005-341604 discloses one example of adata communication system.

Japanese Patent Laid-open No. 2003-244691 discloses a system configuredto time-stamp audio visual data to be used later by oneself.

Published Japanese Translation of a PCT Application Nos. 2004-538681,2004-537193, and No. 2004-538679 disclose propositions for enjoyingbroadcast and recorded programs by extending on the WWW (World WideWeb).

However, no conventional technologies have been proposed that allow auser to access the visual other than himself or herself as desired.

It is therefore an object of the present invention to allow a user toview a sight in field of vision other than himself (a scene seen from amoving body other than himself) as desired and with ease.

DISCLOSURE OF INVENTION

An image display system according to the present invention has a displayapparatus and an image pickup apparatus attached to a moving body toexecute image pickup. The image pickup apparatus has image pickup meansfor executing image pickup; communication means for executing datacommunication with an external device; and control means for executingtransmission control processing for making the communication meanstransmit and output image data obtained by image pick up of the imagepickup means. The display apparatus has display means for executingimage display; communication means for executing data communication withan external device; and control means for executing specificationprocessing for setting specification information for specifying aparticular image pickup apparatus, image request transmission processingfor transmission from the communication means an image request about animage pickup apparatus to be specified on the basis of the specificationinformation, and display processing for receiving through thecommunication means image data transmitted in response to the imagerequest and making the display means execute a display operation basedon the received image data.

The above-mentioned moving body is a human and the above-mentioned imagepickup apparatus is structured to be attached to a human and theabove-mentioned image pickup means is configured to take images of inthe field-of-vision direction of the user wearing the above-mentionedimage pickup apparatus.

Alternatively, the above-mentioned moving body is any of a creatureother than a human, or a ground moving device, or a marine movingdevice, or a submarine moving device, or an air moving device, or aspace moving device.

The above-mentioned communication means of the above-mentioned displayapparatus executes data communication with the communication means of aparticular image pickup apparatus specified on the basis of thespecification information, thereby receiving image data from theparticular image pickup apparatus.

The above-mentioned image display system further has a server apparatuscommunicable with the display apparatus and the image pickup apparatus,wherein image data transmitted from the image pickup apparatus istransmitted to the display apparatus via the server apparatus.

In this case, the above-mentioned display apparatus transmits theabove-mentioned specification information to the above-mentioned serverapparatus along with the above mentioned image request andabove-mentioned server apparatus transmits image data transmitted from aparticular image pickup apparatus specified by above-mentionedspecification information to above-mentioned display apparatus.

Alternatively, the above-mentioned server apparatus specifies aparticular image pickup apparatus on the basis of the above-mentionedspecification information, thereby transmitting image data transmittedfrom the specified image pickup apparatus to the above-mentioned displayapparatus.

The above-mentioned specification information is information indicativeof the above-mentioned image pickup apparatus or a user of theabove-mentioned image pickup apparatus.

Also, the above-mentioned specification information is positionalinformation, or orientation information as image pickup direction, orelevation angle information as image pickup direction, or altitudeinformation of image pickup place, or moving speed information of amoving body at image pickup.

Also, the above-mentioned specification information is informationindicative of performance of the above-mentioned image pickup apparatus.

Also, the above-mentioned specification information is informationindicative of an image pickup operation state of the above-mentionedimage pickup apparatus.

Also, the above-mentioned specification information is informationindicative of a type of the above-mentioned moving body.

Also, the above-mentioned specification information is informationindicative of an image of images taken by a plurality of theabove-mentioned image pickup apparatuses. For example, theabove-mentioned server apparatus transmits each image data taken by aplurality of the above-mentioned image pickup apparatuses to theabove-mentioned display apparatus as image data for specification andthe above-mentioned control means of the above-mentioned displayapparatus uses information indicative of an image selected from theabove-mentioned received image data for specification as theabove-mentioned specification information as the above-mentionedspecification processing. Further, the above-mentioned server apparatusextracts one or more the above-mentioned image pickup apparatuses undera predetermined condition

Also, the above-mentioned specification information includes informationfor specifying a particular image pickup apparatus and time information.

Also, the above-mentioned specification information is information forspecifying one or more particular image pickup apparatuses.

A display apparatus according to the present invention has display meansfor executing image display; communication means for executing datacommunication with an external device; and control means for executingspecification processing for setting specification information forspecifying a particular image pickup apparatus among externalapparatuses attached to a moving body to execute image pickup, imagerequest transmission processing for transmission from theabove-mentioned communication means an image request about an imagepickup apparatus to be specified on the basis of the above-mentionedspecification information, and display processing for receiving throughthe above-mentioned communication means image data transmitted inresponse to the above-mentioned image request and making theabove-mentioned display means execute a display operation based on theabove-mentioned received image data.

The external image pickup apparatus specified on the basis of theabove-mentioned specification information is an image pickup apparatusworn by or mounted on a human, a creature other than a human, or aground moving device, or a marine moving device, or a submarine movingdevice, or an air moving device, or a space moving device as theabove-mentioned moving body.

Also, the above-mentioned control means makes the above-mentionedcommunication means execute data communication with an image pickupapparatus specified on the basis of the above-mentioned specificationmeans.

Also, the above-mentioned control means makes the above-mentionedcommunication means execute data communication with an external serverapparatus so as to receive image data from an image pickup apparatusspecified on the basis of the above-mentioned specification informationvia the above-mentioned server apparatus.

Also, the above-mentioned control means sets the above-mentionedspecification information in response to a specification input operationby a user in the above-mentioned specification processing.

Also, the above-mentioned control means sets information indicative of aparticular image pickup apparatus or a user of the above-mentioned imagepickup apparatus as the above-mentioned specification information in theabove-mentioned specification processing.

Also, the above-mentioned control means sets positional information, ororientation information as image pickup direction, or elevation angleinformation as image pickup direction, or altitude information of imagepickup place, or moving speed information of a moving body at imagepickup as the above-mentioned specification information in theabove-mentioned specification processing.

Also, the above-mentioned control means displays a map image on theabove-mentioned display block and, at the same time, sets positionalinformation in response to a specification input operation by a user forthe above-mentioned map image as the above-mentioned specificationinformation in the above-mentioned specification processing.

Also, the above-mentioned control means displays a map image indicativeof a position of the above-mentioned image pickup apparatus on theabove-mentioned display block and, at the same time, sets informationindicative of a particular image pickup apparatus in response to aspecification input operation by a user for the above-mentioned imagepickup apparatus indicated on the above-mentioned map image as theabove-mentioned specification information.

Also, the above-mentioned control means sets information indicative ofperformance of the above-mentioned image pickup apparatus as theabove-mentioned specification information in the above-mentionedspecification processing.

Also, the above-mentioned control means sets information indicative ofan image pickup operation state of the above-mentioned image pickupapparatus as the above-mentioned specification information in theabove-mentioned specification processing.

Also, the above-mentioned control means sets information indicative of atype of the above-mentioned moving body as the above-mentionedspecification information in the above-mentioned specificationprocessing.

Also, the above-mentioned control means sets information indicative ofan image selected from among images taken by a plurality of theabove-mentioned image pickup apparatuses as the above-mentionedspecification information in the above-mentioned specificationprocessing. In this case, in response to reception by theabove-mentioned communication means of image data for specificationincluding each image data taken by a plurality of the above-mentionedimage pickup apparatuses, the above-mentioned control means makes theabove-mentioned display means display on the basis of theabove-mentioned image data for specification and, in response to aspecification input operation by a user for displaying of theabove-mentioned image data for specification, sets informationindicative of the above-mentioned selected image.

Also, the above-mentioned control means sets the above-mentionedspecification information including time information in theabove-mentioned specification processing.

Also, the above-mentioned control means sets the above-mentionedspecification information for specifying one or more particular imagepickup apparatuses in the above-mentioned specification processing.

Also, the above-mentioned display means has a structure in which theabove-mentioned display means is arranged so as to be located in frontof the above-mentioned eyes of a user for executing image display.

Also, the above-mentioned display apparatus has image pickup means forexecuting an image pickup, wherein the above-mentioned control means canalso execute transmission control processing for making theabove-mentioned communication means transmit and output image dataobtained by image pickup by the above-mentioned image pickup means. Thatis, it can function as an image pickup apparatus in the above-mentionedimage display system.

A display method according to the present invention has a specificationprocessing step for setting specification information for specifying aparticular image pickup apparatus among external image pickupapparatuses configured to be attached to a moving body for executingimage pickup; an image request transmission step for transmitting animage request for an image pickup apparatus to be specified on the basisof the specification information; and a display step for receiving imagedata transmitted in response to the image request and execute display onthe basis of the received image data.

According to the present invention described above, the user of thedisplay apparatus can see field-of-vision sights seen by other than theuser himself. For example, consider, for external image pickupapparatuses, an image pickup apparatus worn by another person, an imagepickup apparatus mounted on automobile, train and so on, and an imagepickup apparatus worn by animal, bird, and so on. And, image data takenby these image pickup apparatuses is transmitted to the displayapparatus and image-displayed on the display apparatus. Then, the userof the display apparatus can see, as display images, an image infield-of-vision sight seen by another person, an image infield-of-vision sight seen from automobile, electric train, and so on,and an image in field-of-vision sight seen by animal, bird and so on.

Namely, by directly or indirectly specifying, from the displayapparatus, one or more particular image pickup apparatuses amongexternal image pickup apparatuses and receiving image data from thespecified image pickup apparatus, the user of the display apparatus cansee field-of-vision sights other than himself that are taken by theexternal image pickup apparatus. In order to directly or indirectlyspecify a particular image pickup apparatus, specification informationis set at the display apparatus. This specification information isinformation such as identification information directly indicative of animage pickup apparatus or the owner thereof or information indicative ofa place of which sight the user of a display apparatus wants to see,situation, moving body type, and image contents. Namely, thespecification information may be information that can eventually specifyone or more certain image pickup apparatuses.

According to the present invention, the user of a display apparatus cansee field-of-vision sights of a moving body other than himself and easysee various sights such as sights that have different viewing locationsand sights that cannot be ordinarily seen. Consequently, the presentinvention provides a system and an apparatus suitable for variousapplications, such as applications providing visual enjoyment, academicstudy applications, information gathering application, and so on.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 shows an example of an external view of an image pickup displayapparatus practiced as an embodiment of the invention.

FIG. 2 is a example of an external view of an image pickup apparatus anda display apparatus practiced as embodiments.

FIG. 3 shows an example of a system configuration practiced as anembodiment.

FIG. 4 an example of a system configuration practiced as an embodiment.

FIG. 5 is an example is a system configuration practiced as anembodiment.

FIG. 6 is a block diagram illustrating the image pickup displayapparatus practiced as the embodiment.

FIG. 7 is a block diagram illustrating the image pickup apparatuspracticed as the embodiment.

FIG. 8 is a block diagram of a display apparatus practiced as theembodiment.

FIG. 9 is a block diagram illustrating the image pickup apparatus andthe display apparatus practiced as the embodiments.

FIG. 10 is a block diagram illustrating a server apparatus practiced asan embodiment.

FIG. 11 shows flowcharts indicative of a system operation example Ipracticed as an embodiment.

FIG. 12 shows images from another moving body displayed on theembodiment.

FIG. 13 shows images from another moving body displayed on theembodiment.

FIG. 14 shows flowcharts indicative of a system operation example IIpracticed as an embodiment.

FIG. 15 shows flowcharts indicative of a system operation example IIIpracticed as an embodiment.

FIG. 16 shows a camera information management table practiced as anembodiment.

FIG. 17 shows images obtained by the specification of map imagespracticed as an embodiment.

FIG. 18 shows flowcharts indicative of a system operation example IVpracticed as an embodiment.

FIG. 19 shows flowcharts indicative of a system operation example Vpracticed as an embodiment.

FIG. 20 shows flowcharts indicative of a system operation example VIpracticed as an embodiment.

FIG. 21 shows flowcharts indicative of a system operation example VIIpracticed as an embodiment.

FIG. 22 shows a system operation example VIII practiced as anembodiment.

FIG. 23 shows flowcharts indicative of a system operation example VIIIpracticed as an embodiment.

FIG. 24 shows specification images and a display image practiced asembodiments.

FIG. 25 shows flowcharts indicative of a system operation example IXpracticed as an embodiment.

FIG. 26 shows a system operation example X practiced as an embodiment.

FIG. 27 shows flowcharts indicative of a system operation example Xpracticed as an embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION

The following describes embodiments of an image display system, adisplay system, and a display method according to the invention. In theembodiments of the invention, an image pickup display apparatus 1 or adisplay apparatus 40 corresponds to a display apparatus as referred toin the claims attached hereto and, as the processing of the image pickupdisplay apparatus 1 or the display apparatus 40, a display methodaccording to the invention is executed. Also, the image pickup displayapparatus 1 or an image pickup apparatus 30 corresponds to an imagepickup apparatus as referred to in claims attached hereto. It should benoted that the image pickup display apparatus 1 practiced as oneembodiment of the invention can therefore function as both a displayapparatus and an image pickup apparatus as referred to in claimsattached hereto.

The description will be done in the following sequence.

External examples of the image pickup display apparatus, the imagepickup apparatus, and the display apparatus

2. Forms of system

3. Configurational examples of the image pickup display apparatus, theimage pickup apparatus, the display apparatus, and the server apparatus

4. System operation example I

5. System operation example II

6. System operation example III

7. System operation example IV

8. System operation example V

9. System operation example VI

10. System operation example VII

11. System operation example VIII

12. System operation example IX

13. System operation example X

14. Effects, variations, and extensions of the invention

External examples of the image pickup display apparatus, the imagepickup apparatus, and the display apparatus

FIG. 1 shows an external example of the image pickup display apparatus 1practiced as an embodiment of the invention. This image pickup displayapparatus 1 is adapted for wearing by a user as an eyeglass-type displaycamera.

The image pickup display apparatus 1 has a mounting unit structured likea frame that semi-circles from one temporal region toward the othertemporal region through the rear of the head and is worn by a user onthis both ear-capsules.

And, in a wearing state as shown in FIG. 1, this image pickup displayapparatus 1 has a structure that a pair of displays panels 2 a, 2 b forthe right eye and the left eye are arranged immediately in front of boththe eyes of the user, namely, at positions where eyeglasses are normallylocated. For these display panels 2 a, 2 b, liquid crystal panels areused for example and the transmissivity is controlled to provide athrough state, namely, transparent state, or translucent state. When thedisplay panels 2 a, 2 b are in the through state, normal wearing likeeyeglasses does not impair user's ordinary life.

Also, an image pickup lens 3 a is arranged such that image pickup lens 3a is directed to the front so as to take an image with the visual sightdirection being as the direction of a subject in a user wearing state.

Further, a light emitting block 4 a for lighting in the direction ofimage taking by the image pickup lens 3 a is arranged on the basis of aLED (Light Emitting Diode) for example.

In addition, although only the left ear side is shown in the figure, apair of earphone speakers 5 a that can be inserted in the user's leftand right auditory canals in the user wearing state is provided.

Besides, microphones 6 a, 6 b are arranged for picking up external soundat the right side of a display block 2 for the right eye and the leftside of the display block 2 for the left eye.

It should be noted that FIG. 1 shows only an example; various otherstructures are possible for the user to wear the image pickup displayapparatus 1. Generally, it may be enough to provide a wearing unit thatis of eyeglass type or head mount type; at least, in the presentembodiment, it may be enough to provide a wearing unit having astructure in which the display panels 2 a, 2 b are arranged in theproximity of the front of the user's eyes and the image pickup directionby the image pickup lens 3 a is oriented in the user's field of vision,namely, in the direction of the front of the user. However, it is alsopossible to make arrangements in which the directions of image pickupare changed and subjects in the rear, upper, left and right sides, anddown (of at feet) can be picked up.

It is also practicable to arrange one display block in correspondencewith one eye, in addition to the configuration in which a pair ofdisplay blocks are arranged in correspondence with both the eyes as thedisplay panels 2 a, 2 b.

It is also practicable to arrange only one earphone speaker 5 a for oneear instead of the earphone speaker 5 a that functions as left and rightstereo speakers. It is also practicable to use only one of microphones 6a, 6 b. Further, the image pickup display apparatus 1 may be configuredto as not to have any microphone or earphone speaker.

It is also possible to provide a configuration in which no lightemitting block 4 a is arranged.

In the above-mentioned example, the image pickup display apparatus 1 hasa wearing unit of eyeglass type or head mount type; it is alsopracticable to be worn by a user by using a wearing unit of headphonetype, neckband type, or ear hung type. In addition, it is practicable toprovide a wearing unit to be worn by the user with a fastener such as aclip onto an ordinary visor or a headphone for example. Also, thiswearing unit may not necessarily be mounted on the head of user.

It should be noted that the image pickup display apparatus 1 shown inFIG. 1 is an example in which a component block for image pickup and thedisplay panels 2 a, 2 b are integrated to be worn by the user; however,as a device worn by the user, an image pickup apparatus 30 and a displayapparatus 40 as shown in (a) and (b) of FIG. 2 can be conceivable.

The image pickup apparatus 30 shown in (a) of FIG. 2 is worn on thetemporal region of head with a predetermined mounting frame. And, so asto do image pickup with the user's field of vision as the direction ofsubject in the wearing state, the image pickup lens 3 a and the lightemitting block 4 a are arranged toward the front. In addition, themicrophone 6 a for picking up external sound is arranged.

Namely, this image pickup apparatus 30, having no display capabilities,provides an apparatus for taking a scene in the user's field of visionwhen worn on the user. It should be noted that, with the image pickupapparatus 30 such as this, various shapes, mounting structures, andcomponent elements are possible as with the above-mentioned image pickupdisplay apparatus 1.

The display apparatus 40 shown in (b) of FIG. 2 is an example of adisplay apparatus of a wrist watch type, in which the display panel 2 ais formed visually recognizable by the user in a state where theapparatus is worn on the wrist of the user with a wrist band.

It should be noted that here is shown the display apparatus 40 of wristwatch type; however, it is supposed that the display apparatus 40 wornby carried by the user have various shapes and mounting structures. Forexample, the display apparatus 40 may be a portable, small device thatis carried by the user. Alternatively, the display apparatus 40 shown inFIG. 1 is possible that is of eyeglass type and can be carried by theuser (an apparatus obtained by removing the image pick up capabilitiesfrom the image pickup display apparatus 1 shown in FIG. 1).

Further, in the case of the display apparatus 40 that can be carriedportable by the user, not only a device dedicated to monitoring displaybut also such devices having display capabilities as a mobile phone, aportable game machine, a PDA (Personal Digital Assistant) may providethe display apparatus 40 of this example.

In addition to devices that are worn or carried by the user, a desktoptype display apparatus, a computer apparatus, a television receiver, andon-vehicle display monitor device may be used for the display apparatus40 of this example.

It is also practicable to use the image pickup apparatus 30 and thedisplay apparatus 40 shown on (a) and (b) of FIG. 2 in a separatemanner; however, a use form may be supposed in which the user wears boththe image pickup apparatus 30 and the display apparatus 40 and uses themas an image pickup display apparatus based on these two devices. Forexample, the image pickup apparatus 30 and the display apparatus 40 mayexecute data communication with each other for the display apparatus 40to monitor images taken by the image pickup apparatus 30 or displayimages transmitted from an external device.

Meanwhile, in the present example, the use of the image pickup displayapparatus 1 and the display apparatus 40 are assumed for use by the user(human); however, the image pickup apparatus 30 is assumed for beingworn by various moving bodies other than humans, moving bodies being ofvarious kinds including humans. The image pickup apparatus 30 as shownin (a) of FIG. 2 is an example in which it is worn by a human and picksup a sight of field of vision of the human. However, it is conceivablethat the image pickup apparatus 30 is mounted to moving bodies otherthan humans.

Moving bodies other than humans may include creatures other than humans,ground moving devices, marine moving devices, submarine moving devices,air moving devices, and space moving devices, for example.

Creatures other than humans may include birds, mammals, reptiles,amphibians, fishes, insects, and others.

Ground moving devices may include power vehicles such as passenger cars,trucks, buses, taxis, and motor bikes, and man-powered vehicles such asbicycles, rickshaws, and toy cars. Railroad vehicles such as electrictrains and steam locomotives are also assumed. In addition, pleasurevehicles in amusement parks and work vehicles in factories and otherfacilities are supposed. Also, unmanned moving bodies are possible. Forexample, these moving bodies may include business or hobby robots andso-called radio-controlled toys.

Marine moving bodies may include various types of ships, water motorbikes, surf boards, rowing boats, floats, and rafts.

Submarine moving bodies may include submarines, submarine robots, anddiving equipment such as aqualungs.

Air moving bodies may include various types of airplanes, helicopters,gliders, parachutes, balloons, and kites.

Space moving bodies may include rockets, space probes, and artificialsatellites.

Although various more specific examples of moving bodies are possiblefor the moving bodies, the image pickup apparatus 30 may only bedesigned in shape and mounting structure suited for each moving body.

2. Forms of System

In the embodiment, the user of the image pickup display apparatus 1 orthe display apparatus 40 can access another image pickup displayapparatus 1 or the image pickup apparatus 30 as desired to see imagestaken by that another image pickup display apparatus 1 or the imagepickup apparatus 30. Namely, the user of the image pickup displayapparatus 1 or the display apparatus 40 can see, on his image pickupdisplay apparatus 1 or the display apparatus 40, images representing ascene that can be seen by the sight in field of vision of another personor above-mentioned various moving bodies.

Exemplary systems forms configured to realize the above capabilities areshown in FIG. 3, FIG. 4, and FIG. 5.

FIG. 3 shows an example in which the image pickup display apparatus 1,the display apparatus 40, and the image pickup apparatus 30 eachdirectly executes data communication as required.

It is assumed here that users A, B, and C wear the image pickup displayapparatuses 1 as shown in FIG. 1 for example. It is also assumed thatuser D wear the display apparatus 40 as shown in (b) of FIG. 2 forexample and user E wear the image pickup apparatus 30 as shown in (a) ofFIG. 2. Further, it is assumed that user F wear both the image pickupapparatus 30 shown in (a) of FIG. 2 and the display apparatus 40 shownin (b) of FIG. 2, which function as an image pickup display apparatus.

In this case, users A, B, C, D, and F having the image pickup displayapparatuses 1 or the display apparatuses 40 can access another device asdesired to see a sight in field of vision of any other users.

For example, user A specifies, from his image pickup display apparatus1, the image pickup display apparatus 1 of user B to access thereto.Then, the image data being taken by the image pickup display apparatus 1of user B is transmitted. The image pickup display apparatus 1 of user Areceives this image data and outputs the received image data fordisplay. Consequently, user A gets in a state where the sight in fieldof vision of user B can be seen.

FIG. 3 shows a similar operation executed between user C and user D,user C and user E, and user E and user F. For example, user D specifies,from his display apparatus 40, the image pickup display apparatus 1 ofuser C to request the transmission of the taken image data. Then, therequested image data is transmitted from the image pickup displayapparatus 1 to the display apparatus 40 to be displayed at the displayapparatus 40. Consequently, user D can see the sight in field of visionof user C.

User E only wears the image pickup apparatus 30 having no displaycapabilities, so that user E cannot see field of vision of the otherusers. In this case, user E is positioned as a provider of images offield-of-vision sight. User C and user F can see the sight in field ofvision of user E by specifying the image pickup apparatus 30 of user E.It should be noted that the image pickup apparatus 30 worn by variousmoving bodies other than the human mentioned above may be thought aspositioned as an image provider as with the image pickup apparatus 30 ofthe user E.

FIG. 4 shows the image pickup display apparatuses 1, the image pickupapparatuses 30, and the display apparatuses 40 owned by users A throughF, in which these devices communicate with each other via a network 60.Each image pickup display apparatus 1 or display apparatus 40 accessesanother image pickup display apparatus 1 or display apparatus 40 by thecommunication via the network 60, thereby requesting images. Then, theimage data transmitted in response to the request is received anddisplayed.

The case shown in FIG. 3 is a system example in which the devicesdirectly communicate with each other; for example, this example issuited for a system used in a comparatively narrow range in which directcommunication is practicable, such as only inside sport facilitiesincluding succor and baseball parks, only inside a theme park, onlyinside an event arena, or only inside a particular district. Obviously,the size of the range depends on the communication capabilities providedto the image pickup display apparatus 1 and so on; therefore, if accessto other devices in wider range, worldwide for example, is required, itis suitable to execute mutual communication via the network 60 as shownin FIG. 4.

It should be noted that the network 60 may be a wide area network, suchas the Internet; obviously, a narrow area network, such as a LAN (LocalArea Network) is also possible.

FIG. 5 shows the image pickup display apparatuses 1, the image pickupapparatuses 30, and the display apparatuses 40 owned by users A throughF, which form a system in which each of these devices executecommunication via a server apparatus 70 on the network 60.

Namely, from the image pickup display apparatus 1 or the displayapparatus 40, access is made to the server apparatus 70, therebyrequesting an image taken by the specified image pickup displayapparatus 1 or the image pickup apparatus 30. The server apparatus 70communicates with the specified image pickup display apparatus 1 or theimage pickup apparatus 30 to request the image and transfers thetransmitted image data to the image pickup display apparatus 1 or thedisplay apparatus 40. This allows the user of the image pickup displayapparatus 1 or the display apparatus 40 to see the image taken byanother image pickup display apparatus 1 or image pickup apparatus 30.

It should be noted that, in the system forms shown in FIG. 3, FIG. 4,and FIG. 5, a user can see the sight of field of vision of another userin a realtime manner (realtime herein does not consider a time lagcaused by communication for example); however, it is also practicablefor a user to see a past sight.

For example, in the cases shown in FIG. 3 and FIG. 4, if the imagepickup apparatus 30 and the image pickup display apparatus 1 have memorylarge enough for storing taken images for a certain period, images takenby the image pickup display apparatus 1 or the image pickup apparatus 30in the past that can be stored in memory can be transmitted to thedisplay apparatus 40 and the image pickup display apparatus 1.

Alternatively, in the case of the system form shown in FIG. 5, if imagestaken by the image pickup display apparatus 1 or the image pickupapparatus 30 are stored in the server apparatus 70, images taken by theimage pickup display apparatus 1 or the image pickup apparatus 30 in thepast can be provided to the display apparatus 40 and the image pickupdisplay apparatus 1.

Further, if the image pickup display apparatus 1 or the displayapparatus 40 specifies an image pickup display apparatus 1 or an imagepickup apparatus 30 to be accessed, the specification of the imagepickup display apparatus 1 or the image pickup apparatus 30 can bedirectly executed by use of apparatus or user identificationinformation; indirect specification is also practicable. For example,the specification can be executed by use of the location, image pickupdirection, elevation angle, altitude, and type of location of a locationof image pickup, the moving speed information at the time of imagepickup, the type of moving body, the performance of the image pickupdisplay apparatus 1 or the image pickup apparatus 30, the image pickupstate of the image pickup display apparatus 1 or the image pickupapparatus 30, or the contents of image pickup being executed. In thiscase, the image pickup display apparatus 1 or the display apparatus 40can realize an operation of accessing the image pickup display apparatus1 or the image pickup apparatus 30 that satisfies these specificationconditions or the server apparatus 70 can realize an operation ofsearching for the image pickup display apparatus 1 or the image pickupapparatus 30 that satisfies these specification conditions for access inthe system form shown in FIG. 5.

3. Configurational Examples of the Image Pickup Display Apparatus, theImage Pickup Apparatus, the Display Apparatus, and the Server Apparatus

The following describes the exemplary configurations of the image pickupdisplay apparatus 1, the image pickup apparatus 30, the displayapparatus 40, and the server apparatus 70 with reference to FIG. 6through FIG. 10.

First, an exemplary configuration of the image pickup display apparatus1 shown in FIG. 6.

A system controller 10, configured by a microcomputer having a CPU(Central Processing Unit), a ROM (Read Only Memory), a RAM (RandomAccess Memory), a nonvolatile memory block, and an interface block,provides a control block configured to control the entire image pickupdisplay apparatus 1. The system controller 10 executes variousoperational processing and transfers control signals for example witheach component block on the basis of programs held in the ROM and so on,thereby making each component block execute operations.

Inside the image pickup display apparatus 1, an image pickup block 3 isarranged as a configuration for taking an image of the sight in thedirection of the field of vision of the user.

The image pickup block 3 has an image pickup optical system, an imagepickup device block, and an image pickup signal processing block.

The image pickup optical system in the image pickup block 3 has a lenssystem based on the image pickup lens 3 a shown in FIG. 1, a stop, azoom lens, and a focus lens and a drive system configured to make thelens system execute a focus operation and a zoom operation.

The image pickup device block in the image pickup block 3 has asolid-state image pickup device array for detecting an image pickuplight obtained by the image pickup optical system to executephotoelectric conversion, thereby generating an image pickup signal. Thesolid-state image pickup device array is a CCD (Charge Coupled Device)sensor array or a CMOS (Complementary Metal Oxide Semiconductor) sensorarray.

The image pickup signal processing block in the image pickup block 3 hasa sample hold/AGC (Automatic Gain Control) circuit for executing gaincontrol and waveform shaping and a video A/D converter, therebyproviding pickup image data that is digital data. Also, the image pickupsignal processing block executes white balance processing, luminanceprocessing, color signal processing, and anti-shake processing on thepickup image data.

An image pickup operation is executed by the image pickup block 3 havingthese image pickup optical system, image pickup device block, and imagepick up signal processing block, thereby obtaining an image data.

The system controller 10 executes control of turning on/off each imagepickup operation in the image pickup block 3, driving the zoom lens andthe focus lens of the image pickup optical system, the sensitivity andframe rate of the image pickup device block, and setting of parametersand execution of the processing of the image pickup signal processingblock.

The pickup image data obtained by an image pickup operation by thisimage pickup block 3 can be supplied to the display block 2, a storageblock 25, and a communication block 26 via an image processing block 15.

Under the control of the system controller 10, the image processingblock 15 executes processing of converting pickup image data into apredetermined image data format and predetermined signal processing formonitor-displaying the image data on the display block 2. The signalprocessing for monitor-displaying images on the display block 2 includesluminance level adjustment, color correction, contrast adjustment,sharpness (highlighting) adjustment, screen partitioning processing,character image synthesis processing, generation of zoom-in or zoom-outimages, and image effect processing such as mosaic image/luminanceinversion image/soft focus/partial highlighting in image/variation ofcolor atmosphere of entire image.

Also, the image processing block 15 transfers image data between theimage pickup block 3, the display block 12, the storage block 25, andthe communication block 26. Namely, the image processing block 15executes the processing of supplying pickup image data from the imagepickup block 3 to the display block 2, the storage block 25, and thecommunication block 26, the processing of supplying the image datareproduced in the storage block 25 to the display block 2, and theprocessing of supplying the image data received through thecommunication block 26 to the display block 2.

For the configuration for displaying to the user in the image pickupdisplay apparatus 1, the display block 2 is arranged. This display block2 has display panel blocks 2 a, 2 b that is, for example, liquid crystalpanels described above, and a display drive block for driving thedisplay panel blocks 2 a, 2 b.

The display drive block is configured by a pixel drive circuit fordisplaying an image signal supplied from the image processing block 15onto the display panels 2 a, 2 b that are liquid crystal panels forexample. Namely, in the display panels 2 a, 2 b, the pixels arranged ina matrix are applied with drive signals based on video signal withpredetermined horizontal/vertical drive timings, thereby executingdisplay. This processing allows the displaying on the image pickup block3 as an image pickup monitor, the displaying an reproduced imagereproduced in the storage block 25, and the displaying of a receivedimage received through the communication block 26 on the display panels2 a, 2 b.

In addition, the display drive block can control the transmissivity ofeach pixel on the display panels 2 a, and 2 b so as to make the panelsin a through state, namely, transparent state, or translucent state.

The system controller 10 executes control of turning on/off (through) ofa display operation in this display block 2, the commanding ofprocessing parameters to the image data to be displayed, control ofscreen area setting, and the commanding of generating characters.

Further, the image pickup display apparatus 1 has an audio input block6, an audio processing block 16, and an audio output block 5.

The audio input block 6 has microphones 6 a, 6 b shown in FIG. 1 and amicrophone amplifier block for amplifying the audio signals obtained bythe microphones 6 a, 6 b, and an A/D converter, thereby outputting audiodata.

The audio data obtained in the audio input block 6 is supplied to theaudio processing block 16.

Under the control of the system controller 10, the audio processingblock 16 controls the transfer of audio data. Namely, the audioprocessing block 16 supplies the audio data obtained in the audio inputblock 6 to the audio output block 5, the storage block 25, and thecommunication block 26. Alternatively, the audio processing block 16supplies the audio data reproduced in the storage block 25 and the audiodata received by the communication block 26 to the audio output block 5.

Also, under the control of the system controller 10, the audioprocessing block 16 executes the processing of volume adjustment, tonequality adjustment, and sound effect processing.

The audio output block 5 has the pair of earphone speakers 5 a shown inFIG. 1 and an amplifier circuit and a D/A converter for that pair ofearphone speakers 5 a.

Namely, the audio data supplied from the audio processing block isconverted through the D/A converter into an analog audio signal and isamplified through the amplifier circuit to be outputted from theearphone speaker 5 a as sound. Consequently, the user can hear externalsound and sound based on the audio data reproduced in the storage block25 and the audio data received by the communication block 26.

It should be noted that the audio output block 5 may be configured by aso-called bone conduction speaker.

The storage block 25 is a member to record and reproduce image data (andaudio data) on a predetermined recording media. For example, the storageblock 25 is realized as a HDD (Hard Disk Drive). Obviously, recordingmedia may include solid-state memories such as a flash memory, a memorycard incorporating a solid-state memory, an optical disk, amagneto-optical disk, and a hologram memory and the storage block 25 mayonly be configured so that recording and reproducing can be executed inaccordance with recording media used.

Under the control of the system controller 10, this storage block 25records the image data (and audio data) obtained by an image pickupoperation on recording media. Namely, the image data supplied throughthe image processing block 15 and the audio data supplied through theaudio processing block 16 are encoded for recording to recording mediaand the encoded data are recorded to recording media.

Also, under the control of the system controller 10, the storage block25 can reproduce the image data and the audio data that are recorded.The reproduced image data is supplied to the display block 2 through theimage processing block 15 and audio data is supplied to the audio outputblock 5 through the audio processing block 16. Further, the reproducedimage data/audio data can also be supplied to the communication block 26as data to be transmitted to external devices.

It should be noted that, in system operation examples III and VII to bedescribed later, processing is executed to specify another image pickupdisplay apparatus 1 or image pickup apparatus 30 on a map; in this case,map image is displayed on the display block 2. It is assumed that a mapdatabase be stored in the storage block 25 for the displaying of thismap image. Obviously, a storage block for storing the map database maybe arranged outside the storage block 25.

The system controller 10 can execute map search and map displayprocessing by use of a map database.

The communication block 26 executes transfers data with externaldevices. External devices may include another image pickup displayapparatus 1, another image pickup apparatus 30, the display apparatus40, the server apparatus 70.

This communication block 26 may be configured to execute networkcommunication via near-distance wireless communication for a networkaccess point for example on the basis of wireless LAN or Bluetooth ormay execute direct wireless communication with external devices havingcorresponding communication capabilities.

To the communication block 26, the pickup image data taken by the imagepickup block 3 is supplied via the image processing block 15. Also, theaudio data obtained in the audio input block 6 is supplied via the audioprocessing block 16. The communication block 26 can encode these imagedata and audio data for communication and modulate the encoded data forwireless transmission for the transmission to external devices. Namely,the communication block 26 can transmit the realtime image data/audiodata currently taken and picked up by the image pickup display apparatus1 to external devices (another image pickup display apparatus 1, thedisplay apparatus 40, and the server apparatus 70).

In addition, the communication block 26 can encode, for communication,the image data and audio data reproduced in the storage block 25 andmodulate the encoded data for wireless transmission for the transmissionto external devices.

Besides, the communication block 26 receives the image data/audio datasupplied from external devices (another image pickup display apparatus1, the image pickup apparatus 30, and the server apparatus 70) anddemodulates the received data to supply the demodulated data to theimage processing block 15 and the audio processing block 16. In thiscase, the received image and audio are outputted by the display block 2and the audio output block 5.

Obviously, the image data/audio data received by the communication block26 may be supplied to the storage block 25 for recording to recordingmedia.

Also, the image pickup display apparatus 1 has a lighting block 4 and alighting control block 14. The lighting block 4 is composed of a lightemitting block 4 a shown in FIG. 1 and FIG. 2 and a lighting circuit fordriving the lighting block 4 a (LED for example). The lighting controlblock 14 makes the lighting block 4 execute a lighting operation on thebasis of a command from the system controller 10.

Mounting the light emitting block 4 a in the lighting block 4 as shownin FIG. 1 or FIG. 2 allows the lighting block 4 to execute a lightingoperation with the image pickup lens 3 a in the direction of a subject.

In this image pickup display apparatus 1, an operation input block 11 isprovided for the user to execute operations.

The operation input block 11 may have controls such as keys and dialsfor example and be configured so as to detect a user operation of a keyand so on or a user's intentional behavior.

In arranging controls, these controls may be formed to execute a poweron/off operation, image pickup related operations (for example, azooming operation and a signal processing instructing operation),display related operations (for example, a display contents selectingoperation and a display adjusting operation), and an operation forspecifying an external device that will be described later.

In the case of detecting a user's behavior, the installation of anacceleration sensor, an angular velocity sensor, a vibration sensor, anda pressure sensor is possible.

For example, the tapping by the user on the image pickup displayapparatus 1 from one side may be detected with an acceleration sensorand a vibration sensor for example and, when the lateral accelerationexceeds a certain value, for example, the system controller 10 mayrecognize the tapping as a user's operation. Also, a configuration inwhich, if the direction of a user's tapping on sides (portionsequivalent to the bows of a pair of eyeglasses), namely, from the rightside or the left side, can be detected, the tapping can be determined tobe a predetermined operation by the user.

In addition, a user's head swiveling or neck shaking operation may bedetected with an acceleration sensor and an angular velocity sensor forthe system controller 10 to recognize a user's operation.

Also pressure sensors may be arranged on the left and right sides(portions equivalent to the bows of a pair of eyeglasses) and so on ofthe image pickup display apparatus 1 and, when the user presses theright side with a finger, a zooming operation in the telescopicdirection may be effected, while, when the user presses the left sidewith a finger, a zooming operation in the wide-angle direction may beeffected.

Further, a configuration as a biological sensor may be provided in whichuser's biological information is detected to recognize as operationinput. The biological information includes pulse rate, heart rate,electrocardiogram information, myoelectric and aspiration information(for example, breath speed, depth, and amount of ventilation),perspiration, GSR (Galvanic Skin Reflex), blood pressure, blood oxygenlevel, skin surface temperature, brain waves (for example, α wave, βwave, θ wave, and σ wave), blood flow change, and eye state.

And, the system controller 10 may be configured to recognize thebiological sensor detected information as user operation input. Forexample, an eye movement (a change in eyesight direction or a wink) maybe assumed as a user's intentional behavior; for example, when threewinks made by the user is detected, it may be determined to be aparticular operation input. Further, it is practicable to detect thewearing or unwearing of the image pickup display apparatus 1 on thebasis of the above-mentioned detection of biological information or thewearing by a particular user and, for the system controller 10, executepower on/off in accordance with that detection.

The operation input block 11 supplies the information thus obtained as acontrol, an acceleration sensor, an angular velocity sensor, a vibrationsensor, a pressure sensor, and a biological sensor to the systemcontroller 10, which detects user operations on the basis of these itemsof information.

The image pickup display apparatus 1 also has a position detecting block12. The position detecting block 12 is a GPS receiving block forexample. The GPS receiving block receives radio wave from a GPS (GlobalPositioning System) artificial satellite and outputs latitude andlongitude information as a current position to the system controller 10.

This position detecting block 12 is provided as for an image pickupdisplay apparatus 1 or a display apparatus 40 to specify another imagepickup display apparatus 1 or image pickup apparatus 30 by position inthe cases of system operation examples III and IV to be described later.

The system controller 10 executes the processing of transmitting thepositional information (longitude and latitude) detected by the positiondetecting block 12 from the communication block 26 to the serverapparatus 70 on a periodical basis, for example.

It should be noted that, for the position detecting block 12, WiFi(Wireless Fidelity) or positional information services provided by amobile phone carrier may be used.

It is also practicable to detect walking speeds (or vehicle speeds inthe case of the installation on automobiles for example), therebycorrecting detected positions.

The following shows a configuration of the image pickup apparatus 30 inFIG. 7. With reference to FIG. 7, components similar to those previouslydescribed with reference to FIG. 6 are denoted by the same referencenumerals and the description thereof will be skipped. An image pickupapparatus 30 shown in FIG. 7 has a configuration in which the displayblock 2 and the audio output block 5, which are the image and audiooutput systems, of the image pickup display apparatus 1 shown in FIG. 6are eliminated.

Namely, this image pickup apparatus 30 can be worn by the user as shownin FIG. 2 and, as carried by various moving bodies as described above,take images by the image pickup block 3 to transmit pickup image datafrom a communication block 26 to an external device or record to astorage block 25.

A system controller 10 executes control of an image pickup operation, acommunication operation, a recording operation, and so on.

An exemplary configuration of a display apparatus 40 is as shown in FIG.8. Here, again, with reference to FIG. 8, components similar to thosepreviously described with reference to FIG. 6 are denoted by the samereference numerals and the description thereof will be skipped. Theimage pickup apparatus 40 shown in FIG. 8 has a configuration in whichthe image pickup block 3 and the audio input block 6, which provide theimage pickup and audio input functions in the image pickup displayapparatus 1 shown in FIG. 6, are eliminated. The lighting block 4 andthe lighting control block 14 that are auxiliary to image pickup are notarranged, either. Also, because the display apparatus 40 does not becomethe image provider side, the position detecting block 12 arranged forthe purpose of specifying an image provider may not be provided, either.

This display apparatus 40 is a device that is worn by the user in a formshown in (b) of FIG. 2, owned by the user, or owned by the user asinstalled in the home or on the car for example, receiving imagedata/audio data through a communication block 26 from an externaldevice. The received image data/audio data is outputted from a displayblock 2 and a audio input block 6 or recorded to a storage block 25.

A system controller 10 executes control of a communication operation, adisplay operation, an audio output operation, a recording operation, andso on.

Meanwhile, it is also practicable to use both the image pickup apparatus30 and the display apparatus 40 shown in (a) and (b) of FIG. 2 asdescribed above, thereby providing the function equal to that of theimage pickup display apparatus 1.

In this case, an exemplary configuration as shown in FIG. 9 is possibleas the image pickup apparatus 30 and the display apparatus 40.

In example shown in FIG. 9, the image pickup apparatus 30 is given asubstantially the same configuration of that of the image pickup displayapparatus 1 shown in FIG. 6. However, no display block 2 is arranged inthe image pickup apparatus 30; a transmitting block 27 is providedinstead.

The transmitting block 27 encodes the image data to be supplied from theimage processing block 15 for display monitoring, so as to betransmitted to the display apparatus 40. Then the encoded data istransmitted to the display apparatus 40.

The display apparatus 40 also has a receiving block 41, a displaycontrol block 42, and a display block 2.

The receiving block 41 executes data communication with a transmittingblock 21 of the image pickup apparatus 30. Then, the receiving block 41receives image data transmitted from the image pickup apparatus 30 todecode the received image data.

The image data decoded in the receiving block 41 is supplied to thedisplay control block 42. The display control block 42 executes signalprocessing, screen partitioning, and character synthesis for displayingimage data to generate an image signal for displaying, supplying thegenerated image signal to a display block 2 having a display panel 2 athat is a liquid crystal panel.

In response to the image signal for displaying, in the display block 2,the pixels arranged in a matrix of the display panel 2 a are appliedwith drive signals based on video signal with predeterminedhorizontal/vertical drive timings, thereby executing display.

In the above-mentioned configuration, the user wearing the image pickupapparatus 30 and the display apparatus 40 like user F shown in FIG. 3,FIG. 4, and FIG. 5, for example, can use these apparatuses in the samemanner as the image pickup display apparatus 1.

The following shows, in FIG. 10, an exemplary configuration of theserver apparatus 70 shown in FIG. 5.

The server apparatus 70 has a server control block 72, a network storageblock 71, a communication block 73, a camera information managing block74, and a map database 75.

The network storage block 71, realized by a HDD for example, is used totemporarily buffer image data/audio data transmitted from the imagepickup apparatus 30 or image pickup display apparatus 1 via a network 60in transferring these data to the display apparatus 40 or the imagepickup display apparatus 1 or store image data/audio data transmittedfrom the image pickup apparatus 30 or the image pickup display apparatus1 via the network 60 for a predetermined period of time, for example.

The communication block 73 execute data communication between the imagepickup display apparatus 1, the image pickup apparatus 30, and thecommunication block 26 of the display apparatus 40 via the network 60.

The server control block 72 executes control necessary for operations asthe server apparatus 70. To be more specific, the server control block72 executes communication operations between the image pickup displayapparatus 1, the image pickup apparatus 30, and the display apparatus 40and the processing of storing image data/audio data into the networkstorage block 71.

The camera information managing block 74 and the map database 75 providemembers necessary executing system operation examples III through VII,and IX to be described later. The camera information managing block 74manages, in response to a system operation to be executed, the currentposition, image pickup direction (orientation), elevation angle, andmoving speed of the image pickup display apparatus 1 and the imagepickup apparatus 30, manages the performance (specifications) and imagepickup operation state of the image pickup display apparatus 1 and theimage pickup apparatus 30, and manages the types of moving bodies.

For example, in the case where the processing of the current positionmanagement/search of the image pickup display apparatus 1 and the imagepickup apparatus 30, a camera information management table as shown in(a) of FIG. 16 to be described later is used to manage the positionalinformation transmitted from time to time from the image pickup displayapparatus 1 and the image pickup apparatus 30. Also, by matching againstthe map information stored in the map database 75, the current positionsof the image pickup display apparatus 1 and the image pickup apparatus30 can be managed in more detail.

Thus, the configurations of the image pickup display apparatus 1, theimage pickup apparatus 30, the display apparatus 40, and serverapparatus 70 have been shown just as examples. It is obvious that theaddition or deletion of various types of component elements is possiblein accordance with actually practiced system operation examples andfunctions. Further, it is obvious that these configurations are subjectto variation in accordance with the types of moving bodies on which theimage pickup apparatus 30 and the image pickup display apparatus 1 areinstalled (mounted) and the types of the display apparatus 40 (forexample, wristwatch type, portable type, and stationary type).

4. System Operation Example I

The following describes system operations to be executed as presentembodiments.

First, system operation example I is described in which, as shown inFIG. 3 or FIG. 4, the image pickup display apparatus 1 or the displayapparatus 40 executes communication with another image pickup displayapparatus 1 or image pickup apparatus 30 to acquire and display imagedata obtained from another image pickup display apparatus 1 or imagepickup apparatus 30 mentioned above.

FIG. 11 shows the processing of apparatus A and apparatus B as systemoperation example I.

Apparatus A denotes the image pickup display apparatus 1 or the displayapparatus 40 shown in FIG. 3 or FIG. 4. Apparatus B denotes the imagepickup display apparatus 1 or the image pickup apparatus 30 shown inFIG. 3 or FIG. 4.

Namely, apparatus A is a device on the side in which apparatus A is usedby a user to receive an image taken by another moving body and thereceived image is displayed, being equivalent to “display apparatus” asreferred to in claims of the present invention. The processing byapparatus A shown in FIG. 11 is the control processing by the systemcontroller 10 of the image pickup display apparatus 1 or the displayapparatus 40; therefore, the processing by a system controller 10 ofthis apparatus A includes “specification processing,” “image requesttransmission processing,” and “display processing” as referred to inclaims of the present invention.

On the other hand, apparatus B is a device on the side in which an imageis provided in response to the specification from apparatus A, or adevice worn/carried on human, creature, vehicle, or other moving bodiesmentioned above, being equivalent to “image pickup apparatus” asreferred to in claims attached hereto. The processing by apparatus Bshown in FIG. 11 is the control processing by the system controller 10of the image pickup display apparatus 1 or the image pickup apparatus30; therefore, the processing by a system controller 10 of apparatus Bincludes “transmission control processing” as referred to in claims ofthe present invention.

It should be noted that the above-mentioned are common between apparatusA and apparatus B shown in FIG. 14, FIG. 15, FIG. 18, FIG. 19, FIG. 20,FIG. 21, FIG. 23, FIG. 25, and FIG. 27 as system operation examples tobe described later.

The following describes processing shown in FIG. 11 as system operationexample I.

On the side of apparatus A, camera specification processing is firstexecuted as step F100. For example, by use of an operation input block11, the user of apparatus A executes an operation of specifying acertain apparatus B as another image pickup display apparatus 1 or imagepickup apparatus 30.

As one example, the user may enter the camera ID (information forindividually identifying the image pickup display apparatus 1 or imagepickup apparatus 30) of above-mentioned another apparatus B or enter theuser ID (information on user identification such as user code or username) of the user (for example, a friend or an image service provider)owning above-mentioned another apparatus B.

Alternatively, the system controller 10 of apparatus A may listapparatuses B accessible from that apparatus A onto a display block 2 tolet the user select any apparatus B from the displayed list; the userexecutes a selecting operation accordingly, thereby establishing cameraID, user ID, and so on.

Then, in response to these user operations, the system controller 10 ofthe apparatus A sets the apparatus B indicated by the specified cameraID or user ID as an access destination. For the setting of an accessdestination, the information may be taken out that identifies the mateof communication, such as so-called telephone number, electronic mailaddress, or URL (Uniform Resource Locator), communication destinationcode dedicated to local area, and so on. To do so, the system controller10 may hold, in a nonvolatile memory or the like internal thereto, theinformation for identification communication destinations incorrespondence with camera ID and so on.

Ranges (ranges of specifiable devices) of apparatuses B accessible fromapparatus A are various. For example, in the case where group IDs areset among groups of friends for example to execute image datacommunication between the image pickup display apparatus 1, the imagepickup apparatus 30, and the display apparatus 40 having the same groupID, another image pickup display apparatus 1 and another image pickupapparatus 30 in the group in which the apparatus A itself is included asseen therefrom may become the apparatus B.

Also, if image data communication concerned is executed inside aparticular area such as particular facilities or a particular region,another image pickup display apparatus 1 and another image pickupapparatus 30 in that area as seen from the apparatus A may become theapparatus B.

Further, if a service provider such as an enterprise or an organizationthat installed many image pickup apparatuses 30 for example on variousmoving bodies executes the image providing service of this example, theimage pickup apparatus 30 for example installed by that service providermay become apparatuses B as seen from the apparatus A.

Obviously, it is possible to set the specifiable ranges of apparatus Bother than group, area, or those provided by service providers or theimage pickup display apparatus 1 or the image pickup apparatus 30 may beset as the apparatus B as required regardless of the above-mentionedlimitations.

The system controller 10 of the apparatus A executes the cameraspecification processing including the detection of user input and thedisplaying for input in step F100 to establish the specification of acertain apparatus B and then the system controller 10 of the apparatus Aexecutes communication connection processing on the specified apparatusB in step F101. Namely, the system controller 10 of the apparatus Aexecutes communication access to the apparatus B specified by the cameraspecification processing, thereby establishing the communicationconnection with the apparatus B. On the side of the apparatus B,communication establishment processing is executed in response to theaccess from the apparatus A in step F200. Then, when the communicationconnection has been established, the side of the apparatus A transmitsimage request information to the apparatus B. The apparatus B acceptsthe image request.

The apparatus B executes image transmission processing in response tothe image request in step F201. Namely, the system controller 10 of theapparatus B transmits the image data obtained by image pickup in theimage pickup block 3 and the audio data picked up by the audio inputblock 6 to the apparatus A from the communication block 26. It should benoted that, on the side of the apparatus B, the image pickup may havebeen executed at a point of time before the acceptance of the imagerequest or the image pickup may start in response to the image requestacceptance.

When the image transmission processing on the side of the apparatus Bstarts, the reception and display of the image data are executed in step102 on the side of the apparatus A. Namely, the system controller 10 ofthe apparatus A supplies the image data (and the audio data) receivedand demodulated by the communication block 26 to the image processingblock 15 and the audio processing block 16, thereby executing thedisplay output on the display block 2 and the audio output from theaudio output block 5.

The system controller 10 of the apparatus A continuously executes thedisplay output (and the audio output) of the received data until an endis determined in step F103. Also, the apparatus B continues the imagepickup and the transmission of the taken image data (and audio data)until the clear of communication is requested by the apparatus A.

Therefore, while these executing operations are on, the user of theapparatus A can see the scene taken by the apparatus B.

The apparatus A determines in step F103 whether the image displaying hasended or not. For example, if the user executes an operation of endingthe displaying through the operation input block 11, it is determinedthat the displaying has ended. Also, it is possible to automaticallyending the reception and displaying after a predetermined duration oftime. Although not shown in the flowchart of FIG. 11, processing may beexecuted in which the apparatus B instructs the apparatus A to end theabove-mentioned execution.

If the end is found in step F103 according to a user operation oranother ending condition, the system controller 10 of the apparatus Agoes to step F104 to transmit a communication clear request from thecommunication block 26 to the apparatus B.

Confirming the reception of the communication clear request as stepF202, the system controller 10 of the apparatus B goes to step F203.

Then, the system controller 10 of the apparatus A and the systemcontroller 10 of the apparatus B execute the clearing of communicationconnection in step F105 and step F203, respectively, thereby ending thesequence of system operations.

By the above-mentioned processing, the user of the apparatus A canspecify a certain apparatus B as desired to see a sight obtained by thespecified apparatus B on the apparatus A worn or owned by him.

Examples of sights that can be seen by the user of apparatus A are shownin FIG. 12 and FIG. 13.

For example, assume that the user of the apparatus A specify the imagepickup display apparatus 1 or the image pickup apparatus 30 worn orowned by a friend. Then, the user of the apparatus A can see afield-of-vision sight seen by the friend. For example, (a) of FIG. 12shows that a sight seen by the friend in a sightseeing area or a resortis displayed on the display block 2 of the image pickup displayapparatus 1 or the display apparatus 40 that is the apparatus A of theuser. Consequently, the user can enjoy a sight of a remote resort forexample.

If the user of the apparatus A stands at a location in an athleticfestival arena and the location of the user hardly gives a good commandof the view of competitions, the user can specify the apparatus B wornby a friend having a good command of view to allow the user of theapparatus A to see sights as seen by the friend as shown in (b) of FIG.12.

If the user of the apparatus A is in a facility and specifies theapparatus B worn by another person or some moving body in that facility,the user can see sights seen by that another person or some moving body.

For example, assume that the user of the apparatus A be in theauditorium of a succor stadium. Also assume that the apparatus B worn bya succor judge be specifiable by the user of the apparatus A. Then, theuser of the apparatus A can see sights seen by the judge as shown in (c)of FIG. 12 while the user of the apparatus A is sitting in theauditorium.

(d) of FIG. 12 shows a sight of a golf practice. For example, assumethat a sight of swinging of the user of the apparatus A be seen by theuser of the apparatus B from the front. Specifying the apparatus B, theuser of the apparatus A can see an image representing his swinging formtaken by the apparatus B and displayed on the display block 2 of theapparatus A worn by him. Namely, the user of the apparatus A can swingwhile seeing his swinging form, which is useful in checking his swingingforms.

of FIG. 13 shows an exemplary image in which the image pickup displayapparatus 1 or the image pickup apparatus 30 as the apparatus B ismounted on a car. The user of the apparatus A can specify a certainapparatus B mounted on a car to receive the image data transmitted fromthe apparatus B whereby the user of the apparatus A can see sights seenon that car on the apparatus A worn or owned by the user.

(b) of FIG. 13 shows an exemplary image in which the image pickupdisplay apparatus 1 or the image pickup apparatus 30 as apparatus B ismounted on a railroad vehicle. The user of apparatus A can specifyapparatus B mounted on a railroad vehicle to receive image datatransmitted from apparatus B, thereby seeing sights seen from thatrailroad vehicle on apparatus A worn or owned by the user thereof.

(c) of FIG. 13 shows an exemplary image in which the image pickupapparatus 30 as apparatus B is mounted on a bird. The user of apparatusA can specify apparatus B mounted on a bird to receive image datatransmitted from apparatus B, thereby seeing sights in field of visionof that bird (for example, a sight in a nest in this case) on apparatusA worn or owned by the user thereof.

For example, as with the above-mentioned examples, the user of apparatusA can see, as desired, sights seen from other people, other movingdevices, other creatures, and various other moving bodies. Namely, theuser of apparatus A can expand his vision on a pseudo basis to seesights seen from moving bodies other than the user of apparatus A.

5. System Operation Example II

The following describes system operation example II with reference toFIG. 14. This is a processing example in which image data is transmittedfrom apparatus B to apparatus A via the server apparatus 70 as shown inFIG. 5. Operations of the server apparatus 70 shown in FIG. 14 may beconsidered as the processing of the server control block 72 shown inFIG. 10.

On the side of apparatus A, camera specification processing is firstexecuted as step F110. The camera specification processing is the sameas that of above-mentioned step F100 shown in FIG. 11.

After entering the specification of a certain apparatus B by the cameraspecification processing of step F110, the system controller 10 ofapparatus A next accesses the server apparatus 70 in step Fill. Namely,a communication connection state with the server apparatus 70 isestablished. Then, at this moment, image request information and cameraspecification information (for example, camera ID or user ID) indicativeof the specified apparatus B are transmitted to the server apparatus 70.

The server control block 72 of the server apparatus 70 establishescommunication connection with apparatus A to accept the image requestand the camera specification information in step S310. Then, the servercontrol block 72 requests the particular apparatus B indicated by thecamera specification information for communication connection, therebyestablishing communication connection. Then, the server control block 72notifies the particular apparatus B of the image request from apparatusA.

The system controller 10 of apparatus B executes the processing ofproviding communication connection state in response to the connectionrequest from the server apparatus 70 in step F210, thereby startingcommunication and accepting the image request.

In apparatus B, image transmission processing is executed in response tothe image request in step F211. Namely, the system controller 10 ofapparatus B transmits image data obtained by image pickup in the imagepickup block 3 and audio data picked up by the audio input block 6 tothe server apparatus 70 from the communication block 26.

In the server apparatus 70, the image data (and the audio data)transmitted from apparatus B to apparatus A as the processing of stepF311.

In apparatus A, in step S112, the reception and displaying of the imagedata transmitted from apparatus B via the server apparatus 70 areexecuted. Namely, the system controller 10 of apparatus A supplies theimage data (and the audio data) received and demodulated by thecommunication block 26 to the image processing block 15 and the audioprocessing block 16, thereby executing display output on the displayblock 2 and audio output on the audio output block 5.

The system controller 10 of apparatus A continuously executes thisdisplay output (and audio output) of the received data until an end isdetermined in step F113. Also, in apparatus B, until a communicationclear request comes from apparatus A, the image pickup and thetransmission of the taken image data (and audio data) are continued.

Therefore, during this period of time, the user of apparatus A can seesights taken on the side of apparatus B.

The system controller 10 of apparatus A determines in step F113 whetheror not image displaying is ended and, if the end is determined inaccordance with a user operation or an end condition, goes to step F114to transmit a communication clear request from the communication block26 to the server apparatus 70.

The server apparatus 70 is continuing the transfer processing of stepF311 until the communication clear request is received. Then, uponreception of the communication clear request, the server apparatus 70goes from step F312 to F313 to transmit a communication clearinstruction to apparatus B.

Upon confirmation of the reception of the communication clearinstruction as step F212, the system controller 10 of apparatus B goesto step F213.

Then, in step F105, the system controller 10 of apparatus A and, in stepF203, the system controller 10 of apparatus B execute the processing ofclearing the communication connection with the server apparatus 70,respectively. In addition, the server control block 72 of the serverapparatus 70 ends the communication between apparatus A and apparatus Band the server processing in step F314. Thus, the system operation comesto an end.

This system operation executed via the server apparatus 70 also allowsthe user of apparatus A to specify a certain B as desired, therebyseeing sights obtained by the specified apparatus B on the apparatus Aworn or owned by the user of apparatus A. For example, images of sightsillustrated in FIG. 12 and FIG. 13 can be seen.

6. System Operation Example III

The following describes system operation example III with reference toFIG. 15. This is a processing example in which, like the above-mentionedsystem operation example II shown in FIG. 14, image data is transmittedfrom apparatus B to apparatus A via the server apparatus 70, being anexample in which, by specification a position from the side of apparatusA, a certain apparatus B can be specified.

On the side of apparatus A, positional specification processing is firstexecuted as step F120. For example, the system controller 10 ofapparatus A executes map displaying by use of a map database stored inthe storage block 25 or the like. Also, in accordance with a useroperation, search for a certain region on map and scroll displaying areexecuted. Consequently, the user can see map images of a particulardistrict and region on the display block 2.

For example, in response to user's execution of geographic name andscale specification and a scroll operation, the system controller 10changes districts and scales to be displayed, thereby displaying a mapimage of a certain district as shown in (a) of FIG. 17.

And, on this map image, in accordance with a user's operation ofspecifying a certain point, positional specification information isgenerated. For example, positional specification information includingvalues of the longitude and latitude of the specified point isgenerated.

Having generated certain positional specification information by thepositional specification processing in step F120, the system controller10 of apparatus A next accesses the server apparatus 70 in step F121.Namely, a communication connection state is established with the serverapparatus 70. Then, at this moment, the image request information andthe positional specification information are transmitted to the serverapparatus 70.

The server control block 72 of the server apparatus 70 establishescommunication connection with apparatus in step F320 and accepts theimage request and the positional specification information.

Next, in step F321, search is executed for an apparatus B (the imagepickup display apparatus 1 or the image pickup apparatus 30) locatednear a particular position indicated by the positional specificationinformation at that point of time.

It is supposed that, for this search, all the image pickup displayapparatuses 1 and image pickup apparatuses 30 that can provide imagesupply sources be each sequentially or periodically transmitting owncurrent positional information (longitude and latitude) to the serverapparatus 70.

In the server apparatus 70, the camera information managing block 74manages the current positional information by relating with that cameraID for each image pickup display apparatus 1 and image pickup apparatus30.

For example, the camera information managing block 74 has a camerainformation management table as shown in (a) of FIG. 16. This camerainformation management table is a table recording current positionalinformation (CP1, CP2 . . . ) corresponding to camera IDs (CM001, CM002. . . ).

Then, every time current positional information is received from eachimage pickup display apparatus 1 or image pickup apparatus 30, thecurrent positional information corresponding to the camera ID of theimage pickup display apparatus 1 or image pickup apparatus 30 that isthe transmission source is updated on this camera information managementtable, thereby grasping the current positions of all the subject imagepickup display apparatuses 1 or image pickup apparatuses 30.

For example, assume that an apparatus B having camera ID “CM001” beperiodically transmitting the current positional information to theserver apparatus 70, then current positional information CP1 of “CM001”is periodically rewritten on the camera information management table.

In step F321, for the longitude and latitude contained in the positionalspecification information, the server control block 72 makes the camerainformation managing block 74 search for the image pickup displayapparatus 1 or the image pickup apparatus 30 currently located near,thereby identifying certain one image pickup display apparatus 1 orimage pickup apparatus 30 as apparatus B.

When a certain image pickup display apparatus 1 or image pickupapparatus 30 has been identified as subject apparatus B this time on thebasis of the positional specification information, the server controlblock 72 requests that apparatus B for communication connection in stepF322, executing the processing of establishing communication connection.And the server control block 72 notifies the apparatus B of the imagerequest from apparatus A.

It should be noted that, although not shown in FIG. 15, it is possible,actually, that there is none of the image pickup display apparatus 1 andthe image pickup apparatus 30 near the position indicated by thepositional specification information at that point of time. In such acase, the server control block 72 notifies apparatus A of the inabilityto search for a corresponding apparatus B and therefore the inability toprovide images. In response, the side of apparatus A represents theinability of image provision to the user and ends the processing.

The system controller 10 of the certain apparatus B that received theconnection request by the processing of the server apparatus 70 in stepF322 executes the processing of providing a communication connectionstate in response to that connection request in step F220, startingcommunication and, at the same time, accepting the image request.

In the apparatus B, image transmission processing is executed in stepF221 in response to the image request. Namely, the system controller 10of apparatus B transmits the image data obtained by the image pickup inthe image pickup block 3 and the audio data picked up by the audio inputblock 6 to the server apparatus 70 from the communication block 26.

In the server apparatus 70, the image data (and the audio data)transmitted from the apparatus B is transferred to the apparatus A asthe processing of step F323.

On the side of apparatus A, the reception and displaying of the imagedata transmitted from the apparatus B via the server apparatus 70 areexecuted in step F122. Namely, the system controller 10 of apparatus Asupplies the image data (and the audio data) received and demodulated bythe communication block 26 to the image processing block 15 and theaudio processing block 16, thereby executing the display output on thedisplay block 2 and the audio output on the audio output block 5.

The system controller 10 of apparatus A continuously executes this datadisplay output (and the audio output) of the received data until an endis determined in step F123. Also, in apparatus B, until a communicationclear request comes from apparatus A, the image pickup and thetransmission of the taken image data (and audio data) are continued.

Therefore, during this period of time, the user of apparatus A can seesights taken on the side of apparatus B.

The system controller 10 of apparatus A determines in step F123 whetheror not image displaying is ended and, if the end is determined inaccordance with a user operation or an end condition, goes to step F124to transmit a communication clear request from the communication block26 to the server apparatus 70.

The server apparatus 70 is continuing the transfer processing of stepF323 until the communication clear request is received. Then, uponreception of the communication clear request, the server apparatus 70goes from step F324 to F325 to transmit a communication clearinstruction to apparatus B.

Upon confirmation of the reception of the communication clearinstruction as step F222, the system controller 10 of apparatus B goesto step F223.

Then, in step F125, the system controller 10 of apparatus A and, in stepF223, the system controller 10 of apparatus B execute the processing ofclearing the communication connection with the server apparatus 70,respectively. In addition, the server control block 72 of the serverapparatus 70 ends the communication between apparatus A and apparatus Band the server processing in step F326. Thus, the system operation comesto an end.

According to such processing, by specifying a given position by use of amap image, the user of apparatus A can see sights obtained by thespecified apparatus B on the apparatus A worn or owned by the user ofapparatus A. For example, images of sights illustrated in FIG. 12 andFIG. 13 can be seen.

Further, specifying positions is suitable when the user wants to check acurrent sight of a particular region. For example, by specifying acertain road, an actual congestion status can be known in detail byseeing sights of congestion situations near the specified road or thedegree of congestion near an certain event arena can be checked byseeing as sights. Also, by seeing the current situation of a certaintourist spot, how much leaves have been turned red or the weathercondition can be checked as actual sights.

It should be noted that, in this operation example, a certain positionis specified on a map; it is also practicable for the user of apparatusA to enter a particular geographic name and address and transmit theentered geographic name and address to the server apparatus 70 aspositional specification information without displaying a map. In thiscase, on the side of the server apparatus 70, in accordance with thegeographic name and address, the position may be identified (forexample, the identification of the longitude and latitude) from the mapdatabase 75 to search for the apparatus B near the position concerned.

Further, it is also possible to specify an unspecified position (thetype of a place), rather than a particular position. For example,unspecified information such as “mountain,” “beach,” “forest district,”“over sea,” “in sea,” or “sky” is used for positional specificationinformation. And, in the server apparatus 70, search can be executed foran apparatus B at the corresponding position.

Also, on the side of apparatus A, information for specifying thecontents other than positional information or information for specifyingtwo or more items of specification contents may be generated.

For example, in addition to positional information, the specificationinformation may include orientation information as an image pickupdirection, elevation angle information as an image pickup direction,altitude angle of image pickup place, and moving speed information ofmoving body at the time of image pickup.

As described above, if positional information is included inspecification information, a sight taken by an apparatus B at thatposition can be seen; however, it is possible that an image pickupdirection (orientation) is not suitable for the user of apparatus A.Therefore, it is suitable to specify an orientation as an image pickupdirection in addition to positional information. For example, a positioncan be specified on a map image as shown in (a) of FIG. 7 and, at thesame time, an orientation as an image pickup direction from thatposition can be specified. The specification of orientation may be madein east, west, south, north, south east, south west, north west, northeast, east south east, south south east, south south west, west southwest, west north west, north north west, north north east, east northeast, or an orientation may be entered by pointing on a map.

And, the apparatus A transmits positional information and orientationinformation to the server apparatus 70 as specification information.

In the server apparatus 70, the camera information managing block 74executes management/search by use of a camera information managementtable as shown in (b) of FIG. 16. In this case, the camera informationmanagement table is a table recording current positional information(CP1, CP2 . . . ) and current image pickup orientations (for example, W:West, SSW: South South West, NE: North East, etc.) corresponding tocamera IDs (CM001, CM002 . . . ).

It is assumed that all the image pickup display apparatuses 1 and theimage pickup apparatuses 30 that can provide image supply sources beeach sequentially or periodically transmitting own current positionalinformation (longitude and latitude) and image pickup directioninformation to the server apparatus 70. It should be noted that, forthis purpose, the image pickup display apparatus 1 and image pickupapparatus 30 have each an orientation sensor in the position detectingblock 12 for example, thereby having to detect image pickup directionstoo.

In the server apparatus 70, every time current positional informationand orientation information are received from each image pickup displayapparatus 1 or image pickup apparatus 30, the current positionalinformation and the current image pickup direction informationcorresponding to the camera ID of the image pickup display apparatus 1or image pickup apparatus 30 that is the transmission source is updatedon this camera information management table, thereby grasping thecurrent positions and image pickup directions of all the subject imagepickup display apparatuses 1 or image pickup apparatuses 30.

For example, assume that an apparatus B having camera ID “CM001” beperiodically transmitting the current positional information and thecurrent orientation information to the server apparatus 70, then currentpositional information CP1 of “CM001” and image pickup directioninformation are periodically rewritten on the camera informationmanagement table.

Then, in the server apparatus 70, when positional information andorientation information are transmitted from apparatus A, search can beexecuted for the corresponding image pickup display apparatus 1 or imagepickup apparatus 30 as the corresponding apparatus B by referencing thecamera information management table, thereby transferring an image fromthe corresponding apparatus B to the apparatus A in the above-mentionedsame processing as that shown in FIG. 15.

Elevation angle information as an image pickup direction, altitudeinformation of image pickup place, and moving speed information ofmoving body at the time of image pickup can be handled in the samemanner.

For example, an elevation angle sensor, an altitude sensor, and avelocity sensor are arranged in the image pickup display apparatus 1 andthe image pickup apparatus 30 and detection information from thesesensors is made ready for sequential transmission to the serverapparatus 70, thereby allowing the server apparatus 70 to grasp theelevation angle, altitude, and moving speed on the camera informationmanagement table for each apparatus B.

In this case, specifying elevation angle, altitude, and moving speedfrom the side of apparatus A allows the server apparatus 70 to searchfor a corresponding apparatus B, thereby transferring an image to theapparatus A in the same processing as that shown in FIG. 15.

Namely, if the server apparatus 70 sequentially updates and managespositional information, orientation information as image pickupdirection, elevation angle information as image pickup direction,altitude information at image pickup place, and moving speed informationof moving body at the time of image pickup for each image pickup displayapparatus 1 and the image pickup apparatus 30, a system operation issupposed in which positional information, orientation information asimage pickup direction, elevation angle information as image pickupdirection, altitude information at image pickup place, and moving speedinformation of moving body at the time of image pickup are usedindividually or in combinations as specification information.

7. System Operation Example IV

The following describes system operation example IV with reference toFIG. 18. This is a processing example in which image data is transmittedfrom apparatus B to apparatus A via the server apparatus 70 in the samemanner as system operation example III shown in FIG. 15, being anexample in which, by specifying camera specifications from apparatus A,a certain apparatus B can be specified.

On the side of apparatus A, camera specifications processing is firstexecuted as step F130. The camera specifications are items associatedwith the performance of the image pickup display apparatus 1 and theimage pickup apparatus 30, including image pickup frame rate,resolution, the effective number of pixels of image pickup device, andimage compression scheme, for example.

For example, the system controller 10 of apparatus A displays a cameraspecifications select screen on the display block 2 to let the user ofapparatus A specify desired camera specifications. It should be notedthat also possible is a specification input scheme for specifying amodel name having desired camera specifications.

Generating specification information as certain camera specificationsinformation by the specification processing of step F130, then in stepF131, the system controller 10 of apparatus A accesses the serverapparatus 70. Namely, a communication connection state is establishedwith the server apparatus 70. And, at this moment, the server apparatus70 is notified of image request information and camera specificationsspecifying information.

The server control block 72 of the server apparatus 70 establishescommunication connection with the apparatus A in step F330, therebyaccepting the image request and the camera specifications specifyinginformation.

Then, in step F331, on the basis of the camera specifications specifyinginformation, the server control block 72 searches for a particular B(the image pickup display apparatus 1 or the image pickup apparatus 30).

For this search, in the server apparatus 70, the camera informationmanaging block 74 manages the specifications information by relatingwith that camera ID for each image pickup display apparatus 1 and imagepickup apparatus 30.

For example, the camera information managing block 74 has a camerainformation management table as shown in (c) of FIG. 16. This camerainformation management table is a table recording specificationsinformation (SP1, SP2 . . . ) corresponding to camera IDs (CM001, CM002. . . ).

In step F331, the server control block 72 makes the camera informationmanaging block 74 search for the corresponding image pickup displayapparatus 1 or the image pickup apparatus 30 on the basis of the cameraspecifications specifying information to identify one image pickupdisplay apparatus 1 or image pickup apparatus 30 as apparatus B.

Identifying a certain image pickup display apparatus 1 or image pickupapparatus 30 as the subject apparatus B of this time, the server controlblock 72 requests that apparatus B for communication connection in stepF332 and executes the processing of establishing communicationconnection. And an image request from the apparatus A is notified.

It should be noted that, although not shown in FIG. 18, it is possible,actually, that there is none of the image pickup display apparatus 1 andthe image pickup apparatus 30 corresponding to the specified cameraspecifications, or that there is only the apparatus impossible tocommunicate. In such a case, the server control block 72 notifiesapparatus A of the inability to search for or communicate with acorresponding apparatus B and therefore the inability to provide images.In response, the side of apparatus A represents the inability of imageprovision to the user and ends the processing.

The system controller 10 of the certain apparatus B that received theconnection request by the processing of the server apparatus 70 in stepF332 executes the processing of providing a communication connectionstate in response to that connection request in step F230, startingcommunication and, at the same time, accepting the image request.

It should be noted that the subsequent processing is the same as thatshown in FIG. 15, so that the description thereof will be omitted.Namely, steps F132 through F135 of apparatus A are the same as stepsF122 through F125 shown in FIG. 15, step F333 through F336 of the serverapparatus 70 are the same as steps F323 through F326, and steps F231through F233 of apparatus B are the same as steps F221 through F223shown in FIG. 15.

According to such processing, by specifying

given camera specifications, the user of apparatus A can see sightsobtained by the apparatus B having the specified performance on theapparatus A worn or owned by the user of apparatus A. Therefore, this isuseful when specifying the image quality as an image from the apparatusB.

It should be noted that it is also practicable to make combinations ofthe specification of camera specifications and the above-mentionedspecification of position, orientation, elevation angle, moving speed,and altitude described in processing example III. For example, anexample is practicable in which an apparatus B having particularspecifications can be specified at a particular position.

8. System Operation Example V

The following describes system operation example V with reference toFIG. 19. This is also the same processing example as above-mentionedsystem operation example III shown in FIG. 15, in which image data istransmitted from apparatus B to apparatus A via the server apparatus 70,being an example in which, by specifying a camera image pickup operationstate from the side of apparatus A, a certain apparatus B can bespecified.

On the side of apparatus A, the processing of specifying an image pickupoperation is first executed as step F160. The image pickup operationstate includes various states as the image pickup operations beingexecuted in the image pickup display apparatus 1 and the image pickupapparatus 30, such as telescopic zoom image pickup, wide-angle zoomimage pickup, high-sensitive image pickup, low-sensitive image pickup,infrared sensitivity enhanced image pickup, ultraviolet sensitivityenhanced image pickup, high frame rate image pickup, monochrome imagepickup/color image pickup, for example.

For example, the system controller 10 of apparatus A displays an imagepickup operation state select image on the display block 2 to let theuser of apparatus A specify desired image pickup operation states.

Having generated specification information for specifying a certainimage pickup operation state by the specification processing in stepF160, the system controller 10 of apparatus A next accesses the serverapparatus 70 in step F161. Namely, a communication connection state isestablished with the server apparatus 70. Then, at this moment, theserver apparatus 70 is notified of image request information and imagepickup operation state specifying information.

The server control block 72 of the server apparatus 70 establishescommunication connection with apparatus A in step F360 and accepts theimage request and the image pickup operation state specifyinginformation.

Then, in step F361, on the basis of the image pickup operation statespecifying information, the server control block 72 searches for aparticular apparatus B (the image pickup display apparatus 1 or theimage pickup apparatus 30).

It is supposed that, for this search, all the image pickup displayapparatuses 1 and image pickup apparatuses 30 which can provide imagesupply sources be each sequentially, periodically, or every time imagepickup operations are switched, transmitting own image pickup operationstate information to the server apparatus 70.

In the server apparatus 70, the camera information managing block 74manages the current positional information by relating with that cameraID for each image pickup display apparatus 1 and image pickup apparatus30.

For example, the camera information managing block 74 has a camerainformation management table as shown in (c) of FIG. 16. This camerainformation management table is a table recording current image pickupoperation states (MST1, MST2 . . . ) corresponding to camera IDs (CM001,CM002 . . . ).

And, every time current image pickup operation state information isreceived from each image pickup display apparatus 1 or image pickupapparatus 30, the image pickup operation state corresponding to thecamera ID of the image pickup display apparatus 1 or the image pickupapparatus 30 that is the source of the transmission is updated on thiscamera information management table, thereby grasping the current imagepickup operation state for all subject image pickup display apparatuses1 or image pickup apparatuses 30.

In step F361, the server control block 72 makes the camera informationmanaging block 74 search for the corresponding image pickup displayapparatus 1 or image pickup apparatus 30 on the basis of image pickupoperation state specifying information to identify one image pickupdisplay apparatus 1 or image pickup apparatus 30 as apparatus B.

Identifying a certain image pickup display apparatus 1 or image pickupapparatus 30 as the subject apparatus B of this time, the server controlblock 72 requests that apparatus B for communication connection in stepF362 and executes the processing of establishing communicationconnection. And an image request from the apparatus A is notified.

It should be noted that, although not shown in FIG. 19, it is possible,actually, that there is none of the image pickup display apparatus 1 andthe image pickup apparatus 30 corresponding to the specified imagepickup operation state, or that there is only the apparatus impossibleto communicate. In such a case, the server control block 72 notifiesapparatus A of the inability to search for or communicate with acorresponding apparatus B and therefore the inability to provide images.In response, the side of apparatus A represents the inability of imageprovision to the user and ends the processing.

The system controller 10 of the certain apparatus B that received theconnection request by the processing of the server apparatus 70 in stepF362 executes the processing of providing a communication connectionstate in response to that connection request in step F260, startingcommunication and, at the same time, accepting the image request.

It should be noted that the subsequent processing is the same as thatshown in FIG. 15, so that the description thereof will be omitted.Namely, steps F162 through F165 of apparatus A are the same as stepsF122 through F125 shown in FIG. 15, step F363 through F366 of the serverapparatus 70 are the same as steps F323 through F326 shown in FIG. 15,and steps F261 through F263 of apparatus B are the same as steps F221through F223 shown in FIG. 15.

According to such processing, by specifying given image pickup operationstate, the user of apparatus A can see sights obtained by the apparatusB being in the specified image pickup operation state on the apparatus Aworn or owned by the user of apparatus A. Therefore, this is useful whenthe user wants to see an image taken in a particular image pickupoperation state as an image from the apparatus B. For example, an imageof the apparatus B that is executing infrared sensitivity enhanced imagepickup (so-called night-vision image pickup).

It should be noted that it is also practicable to make combinations ofthe specification of the image pickup operation state, theabove-mentioned specification of position, orientation, elevation angle,moving speed, and altitude described in processing example III, and theabove-mentioned camera specifications specification described inprocessing example IV. For example, an example is practicable in whichan apparatus B having particular image pickup operation state can bespecified at a particular position.

9. System Operation Example VI

The following describes system operation example VI with reference toFIG. 20, This is also the same processing example as above-mentionedsystem operation example shown in FIG. 15, in which image data istransmitted from apparatus B to apparatus A via the server apparatus 70,being an example in which, by specifying the type of a moving body onwhich the image pickup display apparatus 1 or the image pickup apparatus30 is mounted from the side of apparatus A, a certain apparatus B can bespecified.

On the side of apparatus A, the processing of specifying a moving bodyis first executed as step F190. The system controller 10 of apparatus Adisplays moving body type select screen on the display block 2, such ashuman, automobile, railroad vehicle, airplane, ship, submarine,artificial satellite, bird . . . as a moving body, thereby letting theuser of apparatus A specify desired moving body.

Having generated specification information for specifying a certainmoving body by the specification processing in step F190, the systemcontroller 10 of apparatus A next accesses the server apparatus 70 instep F191 to establish a communication connection state with the serverapparatus 70. Then, at this moment, the server apparatus 70 is notifiedof image request information and moving body specifying information.

The server control block 72 of the server apparatus 70 establishescommunication connection with apparatus A in step F390 and accepts theimage request and the moving body specifying information.

Then, in step F391, on the basis of the moving body specifyinginformation, the server control block 72 searches for a particularapparatus B (the image pickup display apparatus 1 or the image pickupapparatus 30).

In the server apparatus 70, the mounting moving body is managed by acamera information management table for each image pickup displayapparatus 1 and image pickup apparatus 30. Although not shown, such acamera information management table as described with reference to FIG.16 may manage the type information of the moving body by relating witheach camera ID.

In step F391, the server control block 72 makes the camera informationmanaging block 74 search for the corresponding image pickup displayapparatus 1 or image pickup apparatus 30 on the basis of the moving bodyspecifying information to identify one image pickup display apparatus 1or image pickup apparatus 30 as apparatus B.

Identifying a certain image pickup display apparatus 1 or image pickupapparatus 30 as the subject apparatus B of this time, the server controlblock 72 requests that apparatus B for communication connection in stepF392 and executes the processing of establishing communicationconnection. And an image request from the apparatus A is notified.

It should be noted that, although not shown in FIG. 20, it is possible,actually, that there is none of the image pickup display apparatus 1 andthe image pickup apparatus 30 mounted on the specified moving body, orthat there is only the apparatus impossible to communicate. In such acase, the server control block 72 notifies apparatus A of the inabilityto search for or communicate with a corresponding apparatus B andtherefore the inability to provide images. In response, the side ofapparatus A represents the inability of image provision to the user andends the processing.

The system controller 10 of the certain apparatus B that received theconnection request by the processing of the server apparatus 70 in stepF392 executes the processing of providing a communication connectionstate in response to that connection request in step F290, startingcommunication and, at the same time, accepting the image request.

It should be noted that the subsequent processing is the same as thatshown in FIG. 15, so that the description thereof will be omitted.Namely, steps F192 through F195 of apparatus A are the same as stepsF122 through F125 shown in FIG. 15, step F393 through F396 of the serverapparatus 70 are the same as steps F323 through F326 shown in FIG. 15,and steps F291 through F293 of apparatus B are the same as steps F221through F223 shown in FIG. 15.

According to such processing, by specifying a given moving body, theuser of apparatus A can see sights obtained by the apparatus B mountedon the specified moving body on the apparatus A worn or owned by theuser of apparatus A. Therefore, this is useful when the user wants tosee an image from a particular moving body.

It should be noted that it is also practicable to make combinations ofthe specification of moving body, the above-mentioned specification ofposition, orientation, elevation angle, moving speed, and altitudedescribed in processing example III, and the above-mentioned cameraspecifications specification described in processing example IV, and theabove-mentioned specification of image pickup operation state describedin processing example V.

For example, an example is practicable in which the apparatus B mountedon a particular moving body can be specified at a particular position.

Also, for the moving body type specification for use in specificationinformation and search, an example of types, such as human, automobile,airplane, bird . . . , was described; however, various other specifiabletypes are possible. For example, in the case of automobiles, more detailtypes may be specified, such as passenger car, bus, truck, etc., andmodels (model names). In the case of human, the specification by maleand female and by age is possible and the specification of airplane andrailroad vehicle by model is possible. In the case of animals, this alsoholds the same.

10. System Operation Example VII

The following describes system operation example VII with reference toFIG. 21. This uses a map image in the same manner as above-mentionedsystem operation III of FIG. 15, being an operation example in which aparticular image pickup display apparatus 1 or image pickup apparatus 30is specified on a map image.

On the side of apparatus A, positional specification processing is firstexecuted as step F140. For example, the system controller 10 ofapparatus A executes map displaying by use of a map database stored inthe storage block 25 or the like. Also, in response to a user operation,search and scroll displaying are executed on a region on a map.Consequently, the user can see map images of particular region anddistrict on the display block 2.

For example, the user specifies a geographic name and executes a scrolloperation whereby a map image of a certain district as shown in (a) ofFIG. 17 is displayed.

Then, on this map image, in accordance with user's specification of acertain point, positional specifying information is generated. Forexample, position specifying information including the values oflongitude and latitude of the point specified by the user on a map isgenerated.

Having generated certain position specifying information by the positionspecification processing in step F140, the system controller 10 ofapparatus A next accesses the server apparatus 70 in step F141. Namely,a communication connection state is established with the serverapparatus 70. And, at this moment, the server apparatus 70 is notifiedof the position specifying information.

The server control block 72 of the server apparatus 70 establishescommunication connection with the apparatus A and accepts the positionalspecifying information.

Then, in step F341, search is executed for one or more apparatuses B(image pickup display apparatuses 1 or image pickup apparatuses 30)existing near a particular position indicated by the position specifyinginformation at that point of time.

In this case too, it is assumed that, for the search, all the imagepickup display apparatuses 1 and image pickup apparatuses 30 that canprovide image supply sources be each sequentially or periodicallytransmitting own current positional information (longitude and latitude)to the server apparatus 70. In the server apparatus 70, by the camerainformation management table shown in (a) of FIG. 16, the camerainformation managing block 74 manages the current positional informationby relating with that camera ID for each image pickup display apparatus1 and image pickup apparatus 30. Namely, every time current positionalinformation is received from each image pickup display apparatus 1 orimage pickup apparatus 30, the current positional informationcorresponding to the camera ID of the image pickup display apparatus 1or the image pickup apparatus 30 that is the source of the transmissionis updated on this camera information management table, thereby graspingthe current position for all subject image pickup display apparatuses 1or image pickup apparatuses 30.

However, in the case of this operation example VII, not one apparatus Bis identified by search, but all the image pickup display apparatuses 1or image pickup apparatuses 30 existing in a predetermined range aroundthe point indicated by the position specifying information areextracted.

Then, the current positional information of each of the extracted imagepickup display apparatuses 1 and image pickup apparatuses 30 istransmitted to the apparatus A as camera positional information. Forexample, camera IDs and the current positional information thereof ofone or more extracted apparatuses B are transmitted.

It should be noted that, although not shown in FIG. 21, if there is noimage pickup display apparatus 1 or image pickup apparatus 30 around theposition indicated by the position specifying information at that pointof time, the server control block 72 notifies apparatus A of theinability to search for a corresponding apparatus B and therefore theinability to provide images. In response, the side of apparatus Arepresents the inability of image provision to the user and ends theprocessing.

Receiving the camera positional information in step F142, the systemcontroller 10 of apparatus A adds an image of the camera position to themap image displayed in step F140 and displays these images. For example,as shown in (b) of FIG. 17, mark CA of the image pickup displayapparatus 1 or the image pickup apparatus 30 that can provide theapparatus B onto the map image.

For the display as shown in (b) of FIG. 17, the user of apparatus Aexecutes an operation of specifying an apparatus B for image requestamong the apparatuses B appearing on the map.

The system controller 10 of apparatus A accepts the user specificationoperation in step F143 and, at the same time, transmits the camera ID ofthe apparatus B selected by the user to the server apparatus 70 ascamera specifying information, along with the image request.

Receiving the camera specifying information and the image request fromthe apparatus A, the server apparatus 70 requests the apparatus Bcorresponding to that camera specifying information for a communicationconnection in step F342, executing the processing of establishingcommunication connection. Then, the server apparatus 70 transmits theimage request from the apparatus A.

The system controller 10 of an apparatus B that has received aconnection request by the processing of step F342 of the serverapparatus 70 executes the processing of providing a communicationconnection state in accordance with that connection request in stepF240, thereby starting communication and, at the same time, acceptingthe image request.

Then, in the apparatus B, image transmission processing is executed instep F241 in accordance with the image request. Namely, the systemcontroller 10 of the apparatus B transmits the image data obtained bythe image pickup in the image pickup block 3 and the audio data pickedup in the audio input block 6 to the server apparatus 70 from thecommunication block 26.

In the server apparatus 70, the image data (and the audio data)transmitted from the apparatus B is transferred to the apparatus A atthe processing of step F343.

On the side of apparatus A, in step F144, the reception and displayingof the image data transmitted from the apparatus B via the serverapparatus 70 are executed. Namely, the system controller 10 of apparatusA supplies the image data (and the audio data) received and demodulatedby the communication block 26 to the image processing block 15 and theaudio processing block 16, thereby executing the display output on thedisplay block 2 and the audio output on the audio output block 5.

The system controller 10 of apparatus A continuously executes this datadisplay output (and the audio output) of the received data until an endis determined in step F145. Also, in apparatus B, until a communicationclear request comes from apparatus A, the transmission of the imagepickup and the taken image data (and audio data) is continued.

Therefore, during this period of time, the user of apparatus A can seesights taken on the apparatus B specified on the map by the user ofapparatus A.

The system controller 10 of apparatus A determines in step F145 whetheror not image displaying is ended or not and, if the end is determined inaccordance with a user operation or an end condition, goes to step F146to transmit a communication clear request from the communication block26 to the server apparatus 70.

The server apparatus 70 is continuing the transfer processing of stepF343 until the communication clear request is received. Then, uponreception of the communication clear request, the server apparatus 70goes from step F344 to F345 to transmit a communication clearinstruction to apparatus B.

Upon confirmation of the reception of the communication clearinstruction as step F242, the system controller 10 of apparatus B goesto step F243.

Then, in step F147, the system controller 10 of apparatus A and, in stepF243, the system controller 10 of apparatus B execute the processing ofclearing the communication connection with the server apparatus 70,respectively. In addition, the server control block 72 of the serverapparatus 70 ends the communication between apparatus A and apparatus Band the server processing in step F346. Thus, the system operation comesto an end.

According to such processing, the user of apparatus A can specify adesired apparatus B (the image pickup display apparatus 1 or the imagepickup apparatus 30) on a map image and see sights obtained by thespecified apparatus B on the apparatus A worn or owned by the user ofapparatus A. For example, images of sights illustrated in FIG. 12 andFIG. 13 can be seen.

Also, this specifies a particular position, so that, as with the case ofabove-mentioned system operation example III, a request for confirming acurrent sight of a particular area is matched, thereby providing acurrent situation of a particular position more real with images.

It should be noted that, in this operation example, map displaying isfirst executed on the side of apparatus A to specify a certain positionon the map; however, it is also practicable for the user of apparatus Ato enter a particular geographic name or address and transmit theentered geographic name or address to the server apparatus 70 asposition specifying information. In this case, on the side of the serverapparatus 70, a position is identified (for example, the identificationof longitude and latitude) from the map database 75 in accordance withthe geographic name or address, searching for apparatuses B around thatposition. And, the camera IDs and current positional information of oneor more extracted apparatuses B may be transmitted to the apparatus A,thereby allowing the apparatus A to execute the map displaying as shownin (b) of FIG. 17 at that point of time.

Further, in that case, if the server apparatus 70 transmits map imagedata along with the camera IDs and current positional information of oneor more extracted apparatuses B, it becomes unnecessary for the side ofapparatus A to arrange the map database. In other words, in the imagepickup display apparatus 1 or the display apparatus 40 having no mapdatabase, a specifying scheme of specifying a certain image pickupdisplay apparatus 1 or image pickup apparatus 30 on the map image asshown in (b) of FIG. 17 can be realized.

11. System Operation Example VIII

The following describes system operation example VIII. This systemoperation example VIII allows apparatus A to check the contents of animage taken by apparatus B to specify a desired apparatus B.

The system operation will be outlined with reference to FIG. 22. Shownin (a) and (b) of FIG. 22 are the image pickup display apparatus 1 (orthe display apparatus 40) as an apparatus A, the server apparatus 70,and the image pickup display apparatus 1 and the image pickup apparatus30 as apparatuses B.

As shown in (a) of FIG. 22, the apparatus A (the image pickup displayapparatus 1) for example first requests the server apparatus 70 for animage for specification.

In response, the server apparatus 70 transmits an image request to twoor more apparatuses B. Receiving the image request, each apparatus Btransmits image data currently being taken to the server apparatus 70.

The server apparatus 70 generates specification image data on the basisof the image data transmitted from each apparatus B and transmits thespecification image data to the apparatus A. The specification imagedata may be the image data obtained by synthesizing images transmittedfrom two or more apparatuses B into one screen, as shown in (a) of FIG.24, for example.

On the side of apparatus A, the specification image data as shown in (a)of FIG. 24 for example is received and displayed on the display block 2to allow the user to specify and enter a particular image while checkingthe image.

Then, as shown in (b) of FIG. 22, on the side of apparatus A, thespecification information of a certain image is transmitted to theserver apparatus 70 on the basis of the user specification and input. Onthe basis of the specification information for image specification, theserver apparatus 70 identifies the specified apparatus B andcontinuously receives images from the identified apparatus B. Then, thereceived image data is transferred to the apparatus A.

Such an operation allows the user of apparatus A to specify an apparatusB in accordance with the contents of an image being taken, therebyseeing the pickup image sight.

The following describes the processing for such a system operation withreference to FIG. 23.

On the side of apparatus A, the server apparatus 70 is first accessed asstep F170 to establish a communication connection state with the serverapparatus 70. Then, an image for specification request is transmitted tothe server apparatus 70 at this moment.

The server control block 72 of the server apparatus 70 establishescommunication connection with apparatus A in step F370, accepting theimage for specification request.

Then, in step F371, an existing apparatus B is requested forcommunication connection and the processing of establishingcommunication connection is executed. Then, each apparatus B is notifiedof the image request.

In this FIG. 23, the processing by two apparatuses B, apparatus B(1) andapparatus B(2), is representatively shown as apparatus B; the serverapparatus 70 communicably connects to may apparatuses B that arecommunicable at that point of time for example, transmitting imagerequests.

Many apparatuses B(1) apparatus B(2) . . . requested by the serverapparatus 70 for communication each execute the processing of startingcommunication with the server apparatus 70 in step F270, accepting theimage request.

Then, apparatus B(1) apparatus B(2) . . . each starts the processing oftransmitting the image data being taken to the server apparatus 70 instep F271. It should be note that, on the side of apparatus B, the imagepickup may have been executed before the acceptance of image request ormay start image pickup in response to the acceptance of image request.

Also, in the example shown in FIG. 23, description is made as that theapparatus B starts the image data transmission to the server apparatus70 in step F271 in response to the image request from the serverapparatus 70; however, a system operation example is also possible inwhich each apparatus B is always transmitting the pickup image data tothe server apparatus 70.

The server control block 72 of the server apparatus 70 starts receivingimage data from apparatuses B(1) apparatus B(2) . . . in step F372. Andthe server control block 72 starts the processing of the generationprocessing of an image for specification for synthesizing image datafrom apparatuses B(1) apparatus B(2) . . . and the processing oftransmitting the generated image for specification to the apparatus A,for example.

For example, an image for specification for presenting the contents ofimages G1, G2, G3 and G4 as shown in (a) of FIG. 24 is generated byreducing image G1 from each apparatus B(1), image G2 from apparatusB(2), image G3 from apparatus B(3) not shown, and image G4 fromapparatus B(4) not shown and the generated image for specification istransmitted to the apparatus A.

Because moving image pickup data is continuously transmitted from eachapparatus B(1), apparatus B(2) . . . , the server apparatus 70subsequently continues the synthesis processing of the moving imagedata, thereby continuing generation/transmission of the image forspecification including the contents of moving image shown in (a) ofFIG. 24.

It should be noted that images G1, G2, G3, G4 . . . from each apparatusB included in the image for specification may be image data such as apseudo moving image extracted for each predetermined frame or a stillimage of only a frame at the time of the first reception, for example.

Further, images of four apparatuses B are synthesized in FIG. 24;however, it is also practicable to vary image reduction rates orgenerate list images as images of many pages so as to check the pickupimages of more apparatuses B. Still further, without reducing images G1,G2, G3, G4 . . . from each apparatus B, these images may be transmittedto the apparatus A in a time division manner, thereby providing theimages for specification.

The apparatus A starts receiving the image data for specificationtransmitted from the server apparatus 70 in step F171. The systemcontroller 10 of apparatus A starts displaying based on the image datafor specification received at the communication block 26 onto thedisplay block 2.

Consequently, the image as shown in (a) of FIG. 24 is displayed on thedisplay block 2 and the user can confirm the contents of the imagestaken by each apparatus B(1), apparatus B(2) . . . .

The user of apparatus A executes an operation of selecting desired imagecontents by seeing these images. For example, on the display image theuser executes an operation of specifying a desired image. Also, if theimages of all apparatuses B cannot displayed on one screen as the imagesfor specification and therefore the images for specification extend overtwo or more pages, the screen may be fed by user's page feed operationand screen operation.

Then, when the user executes an operation of specifying a certain imagefor the displaying of such an image for specification, the systemcontroller 10 advances the processing from step F171 to step F173 togenerate the specification information indicating the image selected bythe operation, transmitting this specification information and the imagerequest to the server apparatus 70.

Accepting the specification information and the image request from theapparatus A in step F373, the server control block 72 of the serverapparatus 70 advances the processing to step F374.

Namely, the server control block 72 determines a particular apparatus Bby the image specifying information and clears the communication withother apparatuses B. For example, if the image specified by the user ofapparatus A is image G2 shown in (a) of FIG. 24 and the image taken byapparatus B(2), the server control block 72 advances the processing byconsidering that apparatus B(2) has been specified, giving aninstruction of clearing of communication to other apparatuses B (namely,apparatus B(1) and other apparatuses B(3), apparatus B(4) . . . notshown).

In an apparatus B (for example, apparatus B(1)) subject to thetransmission of the communication clear instruction, upon confirmingthat communication clear instruction in step F272, the communicationconnection with the server apparatus 70 is cleared in step F273.

Also, the server control block 72 ends the specification imagegeneration processing in this step F374.

Then, the processing of the server control block 72 goes to step F375 tostart image transfer from the apparatus B being connected. At thismoment for example, only apparatus B(2) for example is kept incommunication connection by the processing of above-mentioned step F374,so that the server apparatus 70 starts the processing of transferringimage G2 from apparatus B to apparatus A.

The apparatus A starts receiving the image data transmitted from theserver apparatus 70 in step F174 and the system controller 10 of theapparatus A starts displaying the image data received at thecommunication block 26 onto the display block 2.

Consequently, image G2 from apparatus B(2) as shown in (b) of FIG. 24for example is displayed on the display block 2. Namely, in response tothe specification by the user of the image data of apparatus B(2) in theimage for specification shown in (a) of FIG. 24 before, a state isprovided in which the image taken by apparatus B(2) can be seen.

The system controller 10 of apparatus A determines in step F175 whetheror not the image displaying is to be ended and, if the end is determinedin accordance with a user operation or an end condition, goes to stepF176 to transmit a communication clear request from the communicationblock 26 to the server apparatus 70.

Until receiving the communication clear request, the server apparatus 70continues the transfer processing of step F375. Then, upon receiving thecommunication clear request, the server apparatus 70 goes from step F376to step F377, transmission a communication clear instruction to theapparatus B (for example, apparatus B(2)) connected for communication.

The system controller 10 of the apparatus B (for example, apparatusB(2)) kept in communication connection so far goes to step F273 uponconfirming the reception of the communication clear instruction as stepF272.

Then, the system controller 10 of apparatus A, in step F177, the systemcontroller 10 of apparatus B, in step F273, each execute thecommunication connection clear processing with the server apparatus 70.Also, the server control block 72 of the server apparatus 70 ends thecommunication between apparatus A and apparatus B and the serverprocessing in step F378. Thus, the system operation comes to an end.

According to such processing, the user of apparatus A can specify adesired apparatus B by use of a technique in which desired imagecontents are selected by seeing image contents of a sight actually beingtaken by many apparatuses B (the image pickup display apparatus 1 or theimage pickup apparatus 30) and see the sight obtained by the specifiedapparatus B on the apparatus A worn or owned by the user of apparatus A.For example, desired images of sights illustrated in FIG. 12 and FIG. 13can be selected and seen.

Especially, because the selection can be executed by image contents, theuser can easily select desired sights.

12. System Operation Example IX

The following describes system operation example IX with reference toFIG. 25. Basically like system operation example VIII mentioned above,this allows the user of apparatus A to confirm an image being taken byeach apparatus B and select desired images, being an example in whichcandidates of apparatuses B can be narrowed down at the stage ofgenerating images for specification in the server apparatus 70.

On the side of apparatus A, setting processing of camera extractioninformation is first executed as step F180.

The camera extraction information is information that provides searchconditions for apparatuses B and, more specifically, includes theinformation of position, orientation, elevation angle, moving speed, andaltitude described in above-mentioned processing example III, theinformation of camera specifications described in above-mentionedprocessing example IV, the information of image pickup operation statedescribed in above-mentioned processing example V, and the informationof moving body types described in above-mentioned processing example VI.

For example, the system controller 10 of apparatus A displays an inputscreen of camera extraction information on the display block 2,requiring the user for selection. For example, a screen on whichposition, orientation, elevation angle, moving speed, altitude, cameraspecifications, image pickup operation state, and moving body type canbe entered is displayed and specification conditions for particularitems are set. Obviously, composite specification conditions such asspecifying both position and moving body type may be set. The systemcontroller 10 sets the camera extraction information in accordance withthe input by the user.

Setting the camera extraction information, the system controller 10accesses the server apparatus 70 in step F181 to establish acommunication connection state with the server apparatus 70. And thesystem controller 10 transmits a specification image request and thecamera extraction information to the server apparatus 70 at this moment.

The server control block 72 of the server apparatus 70 establishescommunication connection with the apparatus A in step F380, acceptingthe specification information request and the camera extractioninformation.

The server control block 72 of the server apparatus 70 executes cameraextraction processing in step F381. For example, the server controlblock 72 makes the camera information managing block 74 execute searchbased on the camera extraction information, extracting one or moreapparatuses B corresponding to the camera extraction information.

For example, if a particular position is indicated as camera extractioninformation, apparatuses B around that position are extracted by thiscamera extraction processing.

Also, if a particular moving body is indicated as camera extractioninformation for example, the apparatus B mounted on that moving body isextracted by this camera extraction processing.

If one or more apparatuses B have been detected on the basis of thecamera extraction information, the server control block 72 requests theextracted apparatus B for communication connection in step F381 toexecute the processing of establishing communication connection. Then,each apparatus B is notified of an image request.

This FIG. 25 also illustrates the processing of two apparatuses B,apparatus B(1) and apparatus B(2), as apparatus B; the server apparatus70 communicably connects to may apparatuses B extracted on the basis ofthe camera extraction information, transmitting image requests.

It should be noted that, although not shown in FIG. 25, if there is noapparatus B corresponding to the camera extraction information or, ifthere is such an apparatus B but that is incommunicable, the servercontrol block 72 notifies apparatus A of the inability to search for orcommunicate with a corresponding apparatus B and therefore the inabilityto provide images. In response, the side of apparatus A represents theinability of image provision to the user and ends the processing.

Apparatus B(1), apparatus B(2) . . . requested by the server apparatus70 for communication each start communication with the server apparatus70 in step F280, accepting an image request.

Then, apparatus B(1), apparatus B(2) . . . each start the processing oftransmitting the image data being taken to the server apparatus 70 instep F281.

The server control block 72 of the server apparatus 70 starts receivingthe image data from apparatus B(1), apparatus B(2) . . . in step F383.Then, the server control block 72 starts the generation processing of animage for specification for synthesizing the image data from apparatusB(1), apparatus B(2) . . . and the processing of transmitting thegenerated image for specification to the apparatus A, for example.

The apparatus A starts receiving the specification image datatransmitted from the server apparatus 70 in step F182. The systemcontroller 10 of apparatus A starts the displaying based on thespecification image data received at the communication block 26 onto thedisplay block 2.

Consequently, the image shown in (a) of FIG. 24 for example is displayedon the display block 2 and the user can confirm the contents of theimages being taken by each apparatus B(1), apparatus B(2) . . . .Especially, in this case, displayed image contents G1, G2, G3, and G4are the images from apparatuses B corresponding to the camera extractioninformation specified by the user.

The user of apparatus A executes an operation of selecting desired imagecontents by seeing this image.

Then, if the user executes an operation of specifying a certain image inresponse to the displaying of such an image for specification, thesystem controller 10 advances the processing from step F183 to F184 togenerate specification information indicative of a specified image andtransmit that specification information and the image request to theserver apparatus 70.

It should be noted that the subsequent processing is the same as thatshown in FIG. 23, so that the description thereof will be omitted.Namely, steps F185 through F188 of apparatus A are the same as stepsF174 through F177 shown in FIG. 23, step F384 through F389 of the serverapparatus 70 are the same as steps F373 through F378 shown in FIG. 23,and steps F282 and F283 of apparatus B are the same as steps F272 andF273 shown in FIG. 23.

According to such processing, the user of apparatus A can specify adesired apparatus B by use of a technique in which desired imagecontents are selected by seeing image contents of a sight actually beingtaken by many apparatuses B (the image pickup display apparatus 1 or theimage pickup apparatus 30) and see the sight obtained by the specifiedapparatus B on the apparatus A worn or owned by the user of apparatus A.

Especially, by narrowing down apparatuses B that are candidates ofspecification by specifying camera extraction information in advance,the image selection that is efficient for the user can be provided andmore efficient system communication and server processing are realized.

13. System Operation Example X

The following describes system operation example X with reference toFIG. 26 and FIG. 27. The above-mentioned system operation examples Ithrough IX are examples in which images taken realtime by the apparatusB can be seen on the apparatus A; however, this system operation exampleV allows the seeing of past images taken by the apparatus B on theapparatus A.

FIG. 26 shows the image pickup display apparatus 1 and the displayapparatus 40 as the apparatus A, the server apparatus 70, and the imagepickup display apparatus 1 and the image pickup apparatus 30 asapparatuses B.

The image pickup apparatus 30 and the image pickup display apparatus 1as apparatuses B are continuously executing the processing oftransmitting taken image data (and audio data) to the server apparatus70. For example, image pickup may always be executed to transmit pickupimage data (and audio data) to the server apparatus 70 or, if imagepickup is executed at particular times, pickup image data (and audiodata) may always be transmitted to the server apparatus 70 at the timeof the execution of image pickup.

The server apparatus 70 stores the image data and the audio datatransmitted from each of the image pickup display apparatus 1 and theimage pickup apparatus 30 into the network storage block 71 by relatingthese data with camera IDs. How long these data are stored in thenetwork storage block 71 may be determined by the size of the networkstorage block 71 and the number of image pickup display apparatuses 1,image pickup apparatuses 30 and so on.

Thus, on the premise that the taken image data (and audio data) arestored in the server apparatus 70 by the image pickup display apparatus1 or the image pickup apparatus 30 that can become an apparatus B, theprocessing shown in FIG. 27 is executed between the apparatus A and theserver apparatus 70.

On the side of apparatus A, camera specification and replay timespecification processing are first executed in step F150. Namely, thesystem controller 10 of apparatus A executes the processing ofspecifying the image pickup display apparatus 1 or one of the imagepickup apparatus 30 as the apparatuses B shown in FIG. 26 in response touser input and specifies a time of desired scene as user input or afixed time. For example, the processing of specifying a time runningback from present like “some minutes before” or the processing ofspecifying a replay time as a particular date and time are executed.

Establishing the specification of a certain apparatus B and thespecification of replay time by the camera specification and relay timespecification processing in step F150, the system controller 10 ofapparatus A next accesses the server apparatus 70 in step F151. Namely,a communication connection state is established with the serverapparatus 70. And, at this moment, the server apparatus 70 is notifiedof information of image request, camera specifying informationindicative of specified apparatus B, and replay time information.

The server control block 72 of the server apparatus 70 establishescommunication connection with the apparatus A in step F350, acceptingthe image request, the camera specifying information, and the replaytime information.

Then, in step F351, a reproduction image is identified. Namely, of theimage data stored in the network storage block 71, the image data takenby the apparatus B specified by the camera specifying information andfrom the date and time specified by the replay time information isidentified as a reproduction image.

Next, the server control block 72 reproduces the image data identifiedas a reproduction image from the network storage block 71 in step F352and transmits the reproduced image data and audio data to the apparatusA from the communication block 73.

On the side of apparatus A, the processing of receiving and displayingthe image data transmitted from the server apparatus 70 is executed instep 152. Namely, the system controller 10 of apparatus A supplies theimage data (and audio data) received and demodulated by thecommunication block 26 to the image processing block 15 and the audioprocessing block 16, thereby executing the display output on the displayblock 2 and the audio output on the audio output block 5.

The system controller 10 of apparatus A continuously executes this datadisplay output (and the audio output) of the received data until an endis determined in step F153. Also, in server apparatus 70, until acommunication clear request comes from apparatus A, the image pickup andthe transmission of the taken image data (and audio data) are continued.

Therefore, during this period of time, the user of apparatus A can seesights of the past taken on the side of apparatus B.

The system controller 10 of apparatus A determines in step F153 whetherimage displaying is ended or not and, if the end is determined inaccordance with a user operation or an end condition, goes to step F154to transmit a communication clear request from the communication block26 to the server apparatus 70.

The server apparatus 70 is continuing the reproduction and transmissionprocessing of step F352 until the communication clear request isreceived. Then, upon reception of the communication clear request, theserver apparatus 70 goes from step F353 to F354.

Then, the system controller 10 of apparatus A executes communicationconnection clear processing with the server apparatus 70 in step F155.Also, the server control block 72 of the server apparatus 70 ends thecommunication with the apparatus A and the server processing in stepF354. Thus, the system operation comes to an end.

Such an operation allows the user of apparatus A to specify a desiredapparatus B and see a past sight obtained by the specified apparatus Bon the apparatus A worn or owned by the user of apparatus A.

For example, assume that each judge wear the image pickup apparatus 30or image pickup display apparatus 1 shown in FIG. 26 in a succor stadiumor a baseball park. Then, sights seen by each judge are accumulated inthe network storage block 71 of the server apparatus 70 as image data.

The user of apparatus A specifies the image pickup apparatus 30 of acertain judge and specifies a replay time as “five minutes before” forexample. Then, the user of apparatus A can see a sight that has beenseen since five minutes before by the judge wearing the specified imagepickup apparatus 30. In this case, the user of apparatus A existing inthe auditorium can see highlight scenes and critical judging scenes forexample in succor or baseball games as a replay image viewed from thejudge.

Especially, in the case where sports are watched in an arena, a succorstadium, and a baseball park, replays cannot be seen as on televisionbroadcasting; however, the use of apparatus A allows the seeing ofreplays as desired, thereby expanding the joy of sport watching.

14. Effects, Variations, and Extensions of Embodiments

In the above, embodiments have been described in which the user of theimage pickup display apparatus 1 or the display apparatus 40corresponding to the above-mentioned apparatus A can see field-of-visionsights of other moving bodies than himself and easily see various sightssuch as sights of different view points and sights that cannot be seenon an usual basis. Consequently, the embodiments of the presentinvention are suited for visual enjoyment, academic studies, and variousinformation gathering activities.

For example, specifying an apparatus B worn by a friend or anacquaintance while traveling allows the seeing of sights currently seenby an acquaintance or the like (or sights at times shortly in the past),thereby also providing a simulated travel experience or the like.

Further, sights that cannot be ordinarily seen, such as sights fromcreature and sights from automobile, ship, airplane, submarine, androcket in places not attended by the user can be seen. This provides anenjoyment of getting a new field of vision and, at the same time, iseffective in information gathering such as road congestion situation andtourist spot situation and ecological studies of creatures.

As described above, interesting field view images can be obtained by thepoints of view other than own point of view, such as the point of viewof a judge and the point of view of a manager at the time of watchingsports such as baseball and succor, thereby widening the joy of sportwatching.

Also, specifying a position or an apparatus B on a map image allows theseeing of sights under the conditions including position. Use of mapimages is suited for the case where the present invention is applied tonavigation apparatuses.

Also, use of position, orientation, elevation angle, moving speed,altitude, camera specifications, image pickup operation state, andmoving body type as specification information allows the more accurateand efficient specification of the apparatus B on which user-desiredimages are taken.

Also, the arrangement of seeing not only realtime images but also replayimages allows the user to experience more various sights.

Also, the image pickup display apparatus 1 or the image pickup apparatus30 that provide the apparatus B is worn by moving bodies includinghumans, so that the provision of fixed facilities such as so-calledfixed-point cameras is made unnecessary.

It should be noted that embodiments of the image display system, thedisplay apparatus, and the display method according to the presentinvention have been described above; however, various variations andextensions are possible in addition to the above-mentioned embodiments.

Configuration examples and processing examples of the image pickupdisplay apparatus 1 or the display apparatus 40 as the apparatus Acorresponding to the display apparatus of the present invention,configuration examples and processing examples of the image pickupdisplay apparatus 1 or the image pickup apparatus 30 as the apparatus Bcorresponding to the image pickup apparatus as cited in claims of thepresent invention, and configuration examples and processing examples ofthe server apparatus 70 are each considered to be various.

For example, although not described in the above-mentioned systemoperation examples I through X, it is also possible to add suchprocessing as rejecting the provision of images to the apparatus A onthe apparatus B specified by the apparatus A.

Also, in system operation examples I through X, the end of communicationis instructed from the apparatus A; however, processing examples arepossible in which communication is ended not by an instruction from theapparatus A but by an instruction from the apparatus B or thecommunication is ended by an instruction from the server apparatus 70.

Also, it is practicable that, on the side of apparatus B, image pickupis executed by not only ordinary image pickup but also various imagepickup operations to transmit taken images or various image pickupoperations are requested from the side of apparatus A. For example,these various image pickup operations include telescopic image pickup,wide-angle image pickup, image pickup while performing zoom-in orzoom-out from telescopic to wide angle, image enlarge pickup, imagereduction pickup, frame-rate variable image pickup (image pickup by highframe rate and low frame rate), high luminance image pickup, lowluminance image pickup, contrast variable image pickup, sharpnessvariable image pickup, sensitivity enhanced image pickup, infraredsensitivity enhanced image pickup, ultraviolet sensitivity enhancedimage pickup, image pickup with particular wavelength band cut, imagepickup applied with image effects such as mosaic processing/luminanceinversion processing/soft focus processing/processing of highlightingpart in image/processing of varying color atmosphere of entire image,and still image pickup.

Also, in the above, image data and audio data are transmitted from theapparatus B to the apparatus A; however, an example is possible in whichonly image data is transmitted.

Alternatively, the present invention may be applied as a system fortransmitting only audio data from the apparatus B to the apparatus A.

Also, assume that a situation occurs in which, when the apparatus A isreceiving the transmission of image data from a certain apparatus B, theapparatus B becomes unable to execute data transmission, then aprocessing example is possible in which, the specification isautomatically switched to another apparatus B to receive image datatransmission. Or it is possible that, in such a case, warning orsituation report are presented to the user.

Also, examples have been described in system operation examples Ithrough X in which one apparatus B is specified and images taken by thespecified apparatus B can be seen on the side of apparatus A; however,it is practicable that two or more apparatuses B are specified in eachsystem operation example.

For example, in the apparatus A, specification information forspecifying two or more particular apparatuses B is generated.

And, in the case of system operation example I, an image request istransmitted to the two or more apparatuses B specified by thespecification information.

Also, in the case of system operation examples II through X, an imagerequest and the specification information for specifying two or moreparticular apparatuses B are transmitted to the server apparatus 70.

Then, the apparatus A receives the image data from the two or moreapparatuses B. In this case, it is practicable to execute operations ofsimultaneously displaying each image data in a screen division mannerand switching these image data for display in response to the useroperation, for example.

Also, in the processing of system operation examples III through IX, inthe server apparatus 70, operations for searching for the apparatus Bare included, on the basis of positional information, orientationinformation, elevation angle information, altitude information, movingspeed information, camera specifications information, image pickupoperation state information, moving body type information, and so on;however, it is practicable to execute the search for the apparatus Bbase on these items of specification information and the camerainformation management for search inside the apparatus A. Namely, theoperations such as system operation examples III through IX can also berealized only by the apparatus A and the apparatus B withoutintervention by the server apparatus 70.

The invention claimed is:
 1. A display apparatus comprising: circuitryconfigured to: transmit specification information for specifying atleast one of a plurality of image pickup apparatuses, each of theplurality of image pickup apparatuses being configured to be attached toa moving body, wherein each of the plurality of image pickup apparatusescan move independently; receive image data from the at least one of theplurality of image pickup apparatuses specified by the specificationinformation in response to transmitting the specification information;and control a display to display the image data received from the atleast one of the plurality of image pickup apparatuses specified by thespecification information, wherein the specification informationincludes moving speed information of a moving body at a time of imagepickup and at least one of positional information, orientationinformation of an image pickup direction, elevation angle information ofthe image pickup direction, and altitude information of an image pickupplace, the specification information being related to each of theplurality of image pickup apparatuses configured to be attached to themoving body.
 2. The display apparatus according to claim 1, wherein thedisplay has a structure in which the display is arranged so as to belocated in front of the eyes of the user for executing image display. 3.The display apparatus according to claim 1, wherein the specificationinformation is information indicative of a particular image pickupapparatus or a user of the image pickup apparatus.
 4. The displayapparatus according to claim 1, wherein the circuitry is furtherconfigured to control the display to display a map image includinglocation information of the at least one of the plurality of imagepickup apparatuses specified by the specification information.
 5. Thedisplay apparatus according to claim 1, wherein the specificationinformation is information indicative of performance of the image pickupapparatus.
 6. The display apparatus according to claim 1, wherein thespecification information is information indicative of a pickupoperation state of the image pickup apparatus.
 7. The display apparatusaccording to claim 1, wherein the specification information isinformation indicative of a type of the image pickup apparatus.
 8. Thedisplay apparatus according to claim 1, wherein the specificationinformation specifies the at least one of the plurality of image pickupapparatuses attached to the moving body.
 9. The display apparatusaccording to claim 1, wherein the specification information includestime information.
 10. The display apparatus according to claim 1,wherein the circuitry is further configured to execute datacommunication with the at least one of the plurality of image pickupapparatuses specified by the specification information.
 11. The displayapparatus according to claim 1, wherein the circuitry is furtherconfigured to execute data communication with an external serverapparatus so as to receive the image data from the at least one of theplurality of image pickup apparatuses specified by the specificationinformation via said server apparatus.
 12. The display apparatusaccording to claim 1, wherein the display is a head mounted displaywearable by a user.
 13. The display apparatus according to claim 1,wherein the specification information specifies the at least one of theplurality of image pickup apparatuses by specifying a moving speed ofthe moving body to which the image pickup apparatus is attached at thetime of the image pickup.
 14. A display method comprising: transmittingspecification information for specifying at least one of a plurality ofimage pickup apparatuses, each of the plurality of image pickupapparatuses being configured to be attached to a moving body, whereineach of the plurality of image pickup apparatuses can moveindependently; receiving image data from the at least one of theplurality of image pickup apparatuses specified by the specificationinformation in response to transmitting the specification information;and displaying, on a display, the image data received from the at leastone of the plurality of image pickup apparatuses specified by thespecification information, wherein the specification informationincludes moving speed information of a moving body at a time of imagepickup and at least one of positional information, orientationinformation of an image pickup direction, elevation angle information ofan image pickup direction, and altitude information of an image pickupplace, the specification information being related to each of theplurality of image pickup apparatuses configured to be attached to themoving body.
 15. The display method according to claim 14, wherein thespecification information is information indicative of a particularimage pickup apparatus or a user of the image pickup apparatus.
 16. Thedisplay method according to claim 14, wherein the displaying displays amap image including location information of the at least one of theplurality of image pickup apparatuses specified by the specificationinformation.
 17. A non-transitory computer readable medium includingexecutable instructions, which when executed by a computer cause thecomputer to execute a display method, the method comprising:transmitting specification information for specifying at least one of aplurality of image pickup apparatuses, each of the plurality of imagepickup apparatuses being configured to be attached to a moving body,wherein each of the plurality of image pickup apparatuses can moveindependently; receiving image data from the at least one of theplurality of image pickup apparatuses specified by the specificationinformation in response to transmitting the specification information;and displaying, on a display, the image data received from the at leastone of the plurality of image pickup apparatuses specified by thespecification information, wherein the specification informationincludes moving speed information of a moving body at a time of imagepickup and at least one of positional information, orientationinformation of an image pickup direction, elevation angle information ofan image pickup direction, and altitude information of an image pickupplace, the specification information being related to each of theplurality of image pickup apparatuses configured to be attached to themoving body.
 18. The non-transitory computer readable medium accordingto claim 17, wherein the displaying displays a map image includinglocation information of the at least one of the plurality of imagepickup apparatuses specified by the specification information.